Trusting Fair Data: Leveraging Quality in Fairness-Driven Data Removal Techniques
Preprint
Abstract
In this paper, we deal with bias mitigation techniques that remove specific data points from the training set to aim for a fair representation of the population in that set. Machine learning models are trained on these pre-processed datasets, and their predictions are expected to be fair. However, such approaches may exclude relevant data, making the attained subsets less trustworthy for further usage. To enhance the trustworthiness of prior methods, we propose additional requirements and objectives that the subsets must fulfill in addition to fairness: (1) group coverage, and (2) minimal data loss. While removing entire groups may improve the measured fairness, this practice is very problematic as failing to represent every group cannot be considered fair. In our second concern, we advocate for the retention of data while minimizing discrimination. By introducing a multi-objective optimization problem that considers fairness and data loss, we propose a methodology to find Pareto-optimal solutions that balance these objectives. By identifying such solutions, users can make informed decisions about the trade-off between fairness and data quality and select the most suitable subset for their application.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Mitigating Representation Bias in Data Transformations: A Constraint-based Optimization Approach2023
- Fairness & friends in the data science eraAI & SOCIETY, 2022
- A Survey on Bias and Fairness in Machine LearningACM Computing Surveys, 2021
- Diversity in Big Data: A ReviewBig Data, 2017
- Fairness Beyond Disparate Treatment & Disparate Impact2017
- Measuring discrimination in algorithmic decision makingData Mining and Knowledge Discovery, 2017
- A data-driven approach to predict the success of bank telemarketingDecision Support Systems, 2014
- Building Classifiers with Independency Constraints2009
- An Improved Dimension-Sweep Algorithm for the Hypervolume Indicator2006
- A fast and elitist multiobjective genetic algorithm: NSGA-IIIEEE Transactions on Evolutionary Computation, 2002