Looking for Trouble: Analyzing Classifier Behavior via Pattern Divergence
Authors:
Open Access
- 9 June 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1400-1412
Abstract
Machine learning models may perform differently on different data subgroups, which we represent as itemsets (i.e., conjunctions of simple predicates). The identification of these critical data subgroups plays an important role in many applications, for example model validation and testing, or evaluation of model fairness. Typically, domain expert help is required to identify relevant (or sensitive) subgroups. We propose the notion of divergence over itemsets as a measure of different classification behavior on data subgroups, and the use of frequent pattern mining techniques for their identification. A quantification of the contribution of different attribute values to divergence, based on the mathematical foundations provided by Shapley values, allows us to identify both critical and peculiar behaviors of attributes. Extended experiments show the effectiveness of the approach in identifying critical subgroup behaviors.Keywords
This publication has 10 references indexed in Scilit:
- An Intersectional Definition of Fairness2020
- FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning2019
- AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic biasIBM Journal of Research and Development, 2019
- Assessing and Remedying Coverage for a Given Dataset2019
- Errudite: Scalable, Reproducible, and Testable Error Analysis2019
- A Survey of Methods for Explaining Black Box ModelsACM Computing Surveys, 2018
- TFX2017
- "Why Should I Trust You?"2016
- Visual exploration of machine learning results using data cube analysis2016
- Mining frequent patterns without candidate generation2000