CNN Fixations: An Unraveling Approach to Visualize the Discriminative Image Regions
- 16 November 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 28 (5) , 2116-2125
Abstract
Deep convolutional neural networks (CNNs) have revolutionized the computer vision research and have seen unprecedented adoption for multiple tasks, such as classification, detection, and caption generation. However, they offer little transparency into their inner workings and are often treated as black boxes that deliver excellent performance. In this paper, we aim at alleviating this opaqueness of CNNs by providing visual explanations for the network's predictions. Our approach can analyze a variety of CNN-based models trained for computer vision applications, such as object recognition and caption generation. Unlike the existing methods, we achieve this via unraveling the forward pass operation. The proposed method exploits feature dependencies across the layer hierarchy and uncovers the discriminative image locations that guide the network's predictions. We name these locations CNN fixations, loosely analogous to human eye fixations. Our approach is a generic method that requires no architectural changes, additional training, or gradient computation, and computes the important image locations (CNN fixations). We demonstrate through a variety of applications that our approach is able to localize the discriminative image locations across different network architectures, diverse vision tasks, and data modalities.Keywords
All Related Versions
Funding Information
- Qualcomm
This publication has 27 references indexed in Scilit:
- On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance PropagationPLOS ONE, 2015
- Going deeper with convolutions2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Caffe2014
- Multiscale Combinatorial Grouping2014
- BING: Binarized Normed Gradients for Objectness Estimation at 300fps2014
- Selective Search for Object RecognitionInternational Journal of Computer Vision, 2013
- How do humans sketch objects?ACM Transactions on Graphics, 2012
- Accurate Object Localization with Shape Masks2007
- Long Short-Term MemoryNeural Computation, 1997