Considerations When Learning Additive Explanations for Black-Box Models
Authors:
- Sarah TanSTSarah Tan
- Giles HookerGHGiles Hooker
- Paul KochPKPaul Koch
- Albert GordoAGAlbert Gordo
- Rich CaruanaRCRich Caruana
Preprint
Abstract
Many methods to explain black-box models, whether local or global, are additive. In this paper, we study global additive explanations for non-additive models, focusing on four explanation methods: partial dependence, Shapley explanations adapted to a global setting, distilled additive explanations, and gradient-based explanations. We show that different explanation methods characterize non-additive components in a black-box model's prediction function in different ways. We use the concepts of main and total effects to anchor additive explanations, and quantitatively evaluate additive and non-additive explanations. Even though distilled explanations are generally the most accurate additive explanations, non-additive explanations such as tree explanations that explicitly model non-additive components tend to be even more accurate. Despite this, our user study showed that machine learning practitioners were better able to leverage additive explanations for various tasks. These considerations should be taken into account when considering which explanation to trust and use to explain black-box models.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- GLocalX - From Local to Global Explanations of Black Box AI ModelsArtificial Intelligence, 2021
- Causal Interpretations of Black-Box ModelsJournal of Business & Economic Statistics, 2019
- Stop explaining black box machine learning models for high stakes decisions and use interpretable models insteadNature Machine Intelligence, 2019
- Network Dissection: Quantifying Interpretability of Deep Visual Representations2017
- Sobol' Indices and Shapley ValueSIAM/ASA Journal on Uncertainty Quantification, 2014
- Explaining prediction models and individual predictions with feature contributionsKnowledge and Information Systems, 2013
- Accurate intelligible models with pairwise interactions2013
- Intelligible models for classification and regression2012
- Fast Stable Restricted Maximum Likelihood and Marginal Likelihood Estimation of Semiparametric Generalized Linear ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2010
- Discovering additive structure in black box functions2004