Improving Interpretability of Deep Neural Networks with Semantic Information
Authors:
- Yinpeng DongYDYinpeng Dong
- Hang SuHSHang Su
- Jun ZhuJZJun Zhu
- Bo ZhangBZBo Zhang
- 1 July 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 975-983
Abstract
Interpretability of deep neural networks (DNNs) is essential since it enables users to understand the overall strengths and weaknesses of the models, conveys an understanding of how the models will behave in the future, and how to diagnose and correct potential problems. However, it is challenging to reason about what a DNN actually does due to its opaque or black-box nature. To address this issue, we propose a novel technique to improve the interpretability of DNNs by leveraging the rich semantic information embedded in human descriptions. By concentrating on the video captioning task, we first extract a set of semantically meaningful topics from the human descriptions that cover a wide range of visual concepts, and integrate them into the model with an interpretive loss. We then propose a prediction difference maximization algorithm to interpret the learned features of each neuron. Experimental results demonstrate its effectiveness in video captioning using the interpretable features, which can also be transferred to video action recognition. By clearly understanding the learned features, users can easily revise false predictions via a human-in-the-loop procedure.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks2016
- Learning Deep Features for Discriminative Localization2016
- Guest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New TechniqueIEEE Transactions on Medical Imaging, 2016
- Sequence to Sequence -- Video to Text2015
- Describing Videos by Exploiting Temporal Structure2015
- Going deeper with convolutions2015
- Translating Videos to Natural Language Using Deep Recurrent Neural Networks2015
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation2014
- Meteor Universal: Language Specific Translation Evaluation for Any Target Language2014
- A Bayesian Hierarchical Model for Learning Natural Scene Categories2005