Mitigating Label Bias in Machine Learning: Fairness through Confident Learning
Open Access
- 24 March 2024
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (15) , 16917-16925
Abstract
Discrimination can occur when the underlying unbiased labels are overwritten by an agent with potential bias, resulting in biased datasets that unfairly harm specific groups and cause classifiers to inherit these biases. In this paper, we demonstrate that despite only having access to the biased labels, it is possible to eliminate bias by filtering the fairest instances within the framework of confident learning. In the context of confident learning, low self-confidence usually indicates potential label errors; however, this is not always the case. Instances, particularly those from underrepresented groups, might exhibit low confidence scores for reasons other than labeling errors. To address this limitation, our approach employs truncation of the confidence score and extends the confidence interval of the probabilistic threshold. Additionally, we incorporate with co-teaching paradigm for providing a more robust and reliable selection of fair instances and effectively mitigating the adverse effects of biased labels. Through extensive experimentation and evaluation of various datasets, we demonstrate the efficacy of our approach in promoting fairness and reducing the impact of label bias in machine learning models.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- LongReMix: Robust learning with high confidence samples in a noisy label environmentPattern Recognition, 2022
- Confident Learning: Estimating Uncertainty in Dataset LabelsJournal of Artificial Intelligence Research, 2021
- A generalized Catoni’s M-estimator under finite α-th moment assumption with α∈(1,2)Electronic Journal of Statistics, 2021
- Bias Mitigation Post-processing for Individual and Group Fairness2019
- The accuracy, fairness, and limits of predicting recidivismScience Advances, 2018
- Challenging the empirical mean and empirical variance: A deviation studyAnnales de l'Institut Henri Poincaré, Probabilités et Statistiques, 2012
- Data preprocessing techniques for classification without discriminationKnowledge and Information Systems, 2011
- Consumer credit-risk models via machine-learning algorithmsJournal of Banking & Finance, 2010
- Evaluating the Predictive Validity of the Compas Risk and Needs Assessment SystemCriminal Justice and Behavior, 2008