Toward Building Trust in Machine Learning Models: Quantifying the Explainability by SHAP and References to Human Strategy
Open Access
- 28 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 11010-11023
Abstract
Local model-agnostic Explainable Artificial Intelligence (XAI), such as LIME or SHAP, has recently gained popularity among researchers and data scientists for explaining black box Machine Learning (ML) models. In the industry, practitioners focus not only on how these explanations can validate their models but also on how they can help maintain trust from end-users. Some studies attempted to measure this ability by quantifying what they refer to as the explainability or interpretability of ML models. In this paper, we introduce a new method for measuring explainability with reference to an approximated human model. We develop a human-friendly interface to strategically collect human decision-making and translate it into a set of logical rules and intuitions, or simply annotations. These annotations are then compared with the local explanations derived from common XAI tools. Through a human survey, we demonstrate that it is possible to quantify human intuition and empirically compare it to a given explanation, enabling a practical quantification of explainability. By relying on this new method, we identified several potential flaws in today’s ML selection process. Furthermore, we demonstrate how our method can help to better evaluate ML models.Keywords
This publication has 11 references indexed in Scilit:
- Quantitative Evaluation of Machine Learning Explanations: A Human-Grounded Benchmark2021
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AIInformation Fusion, 2020
- Quantifying Model Complexity via Functional Decomposition for Better Post-hoc Interpretability2020
- From local explanations to global understanding with explainable AI for treesNature Machine Intelligence, 2020
- Machine Learning Interpretability: A Survey on Methods and MetricsElectronics, 2019
- The Mythos of Model InterpretabilityQueue, 2018
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization2017
- “Why Should I Trust You?”: Explaining the Predictions of Any Classifier2016
- The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clientsExpert Systems with Applications, 2009
- 17. A Value for n-Person Games1953