Quantifying Explainability of Saliency Methods in Deep Neural Networks
Preprint
Abstract
One way to achieve eXplainable artificial intelligence (XAI) is through the use of post-hoc analysis methods. In particular, methods that generate heatmaps have been used to explain black-box models, such as deep neural network. In some cases, heatmaps are appealing due to the intuitive and visual ways to understand them. However, quantitative analysis that demonstrates the actual potential of heatmaps have been lacking, and comparison between different methods are not standardized as well. In this paper, we introduce a synthetic dataset that can be generated adhoc along with the ground-truth heatmaps for better quantitative assessment. Each sample data is an image of a cell with easily distinguishable features, facilitating a more transparent assessment of different XAI methods. Comparison and recommendations are made, shortcomings are clarified along with suggestions for future research directions to handle the finer details of select post-hoc analysis methods.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Understanding the role of individual units in a deep neural networkProceedings of the National Academy of Sciences, 2020
- There and Back Again: Revisiting Backpropagation Saliency Methods2020
- Explainable AI for medical imaging: deep-learning CNN ensemble for classification of estrogen receptor status from breast MRI2020
- Interpretable classification of Alzheimer’s disease pathologies with a convolutional neural network pipelineNature Communications, 2019
- Stop explaining black box machine learning models for high stakes decisions and use interpretable models insteadNature Machine Intelligence, 2019
- Interpreting Deep Visual Representations via Network DissectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Interpretable Explanations of Black Boxes by Meaningful Perturbation2017
- Evaluating the Visualization of What a Deep Neural Network Has LearnedIEEE Transactions on Neural Networks and Learning Systems, 2016
- "Why Should I Trust You?"2016
- Learning Deep Features for Discriminative Localization2016