Probably Approximate Shapley Fairness with Applications in Machine Learning
Open Access
- 26 June 2023
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (5) , 5910-5918
Abstract
The Shapley value (SV) is adopted in various scenarios in machine learning (ML), including data valuation, agent valuation, and feature attribution, as it satisfies their fairness requirements. However, as exact SVs are infeasible to compute in practice, SV estimates are approximated instead. This approximation step raises an important question: do the SV estimates preserve the fairness guarantees of exact SVs? We observe that the fairness guarantees of exact SVs are too restrictive for SV estimates. Thus, we generalise Shapley fairness to probably approximate Shapley fairness and propose fidelity score, a metric to measure the variation of SV estimates, that determines how probable the fairness guarantees hold. Our last theoretical contribution is a novel greedy active estimation (GAE) algorithm that will maximise the lowest fidelity score and achieve a better fairness guarantee than the de facto Monte-Carlo estimation. We empirically verify GAE outperforms several existing methods in guaranteeing fairness while remaining competitive in estimation accuracy in various ML scenarios using real-world datasets.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Data Shapley Valuation for Efficient Batch Active Learning2022
- Improving polynomial estimation of the Shapley value by stratified random sampling with optimum allocationComputers & Operations Research, 2017
- Joint multi-grain topic sentiment: modeling semantic aspects for online reviewsInformation Sciences, 2016
- Learned lessons in credit card fraud detection from a practitioner perspectiveExpert Systems with Applications, 2014
- Computational Aspects of Cooperative Game TheorySynthesis Lectures on Artificial Intelligence and Machine Learning, 2011
- Polynomial calculation of the Shapley value based on samplingComputers & Operations Research, 2009
- Multilinear Extensions of GamesManagement Science, 1972
- A characterization, existence proof and dimension bounds for the kernel of a gamePacific Journal of Mathematics, 1966