Transfer of Machine Learning Fairness across Domains
Preprint
Abstract
If our models are used in new or unexpected cases, do we know if they will make fair predictions? Previously, researchers developed ways to debias a model for a single problem domain. However, this is often not how models are trained and used in practice. For example, labels and demographics (sensitive attributes) are often hard to observe, resulting in auxiliary or synthetic data to be used for training, and proxies of the sensitive attribute to be used for evaluation of fairness. A model trained for one setting may be picked up and used in many others, particularly as is common with pre-training and cloud APIs. Despite the pervasiveness of these complexities, remarkably little work in the fairness literature has theoretically examined these issues. We frame all of these settings as domain adaptation problems: how can we use what we have learned in a source domain to debias in a new target domain, without directly debiasing on the target domain as if it is a completely new problem? We offer new theoretical guarantees of improving fairness across domains, and offer a modeling approach to transfer to data-sparse target domains. We give empirical results validating the theory and showing that these modeling approaches can improve fairness metrics with less data.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Mitigating Unwanted Biases with Adversarial Learning2018
- Measuring and Mitigating Unintended Bias in Text Classification2018
- Towards Robust and Privacy-preserving Text Representations2018
- A survey of transfer learningJournal of Big Data, 2016
- A theory of learning from different domainsMachine Learning, 2009
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009
- Learning from Multiple Sources2007