Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks
Authors:
Preprint
Abstract
As Deep Neural Networks (DNNs) have demonstrated superhuman performance in a variety of fields, there is an increasing interest in understanding the complex internal mechanisms of DNNs. In this paper, we propose Relative Attributing Propagation (RAP), which decomposes the output predictions of DNNs with a new perspective of separating the relevant (positive) and irrelevant (negative) attributions according to the relative influence between the layers. The relevance of each neuron is identified with respect to its degree of contribution, separated into positive and negative, while preserving the conservation rule. Considering the relevance assigned to neurons in terms of relative priority, RAP allows each neuron to be assigned with a bi-polar importance score concerning the output: from highly relevant to highly irrelevant. Therefore, our method makes it possible to interpret DNNs with much clearer and attentive visualizations of the separated attributions than the conventional explaining methods. To verify that the attributions propagated by RAP correctly account for each meaning, we utilize the evaluation metrics: (i) Outside-inside relevance ratio, (ii) Segmentation mIoU and (iii) Region perturbation. In all experiments and metrics, we present a sizable gap in comparison to the existing literature.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Unmasking Clever Hans predictors and assessing what machines really learnNature Communications, 2019
- Pixel Objectness: Learning to Segment Generic Objects Automatically in Images and VideosIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Methods for interpreting and understanding deep neural networksDigital Signal Processing, 2018
- Explaining nonlinear classification decisions with deep Taylor decompositionPattern Recognition, 2017
- DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object DetectionIEEE Transactions on Image Processing, 2016
- Visualizing Deep Convolutional Neural Networks Using Natural Pre-imagesInternational Journal of Computer Vision, 2016
- On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance PropagationPLOS ONE, 2015
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- ImageNet Auto-Annotation with Segmentation PropagationInternational Journal of Computer Vision, 2014