VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 21374-21383
Abstract
Breakthroughs in transformer-based models have revolutionized not only the NLP field, but also vision and multimodal systems. However, although visualization and interpretability tools have become available for NLP models, internal mechanisms of vision and multimodal transformers remain largely opaque. With the success of these transformers, it is increasingly critical to understand their inner workings, as unraveling these black-boxes will lead to more capable and trustworthy models. To contribute to this quest, we propose VL-InterpreT, which provides novel interactive visualizations for interpreting the attentions and hidden representations in multimodal transformers. VL-InterpreT is a task agnostic and integrated tool that (1) tracks a variety of statistics in attention heads throughout all layers for both vision and language components, (2) visualizes cross-modal and intra-modal attentions through easily readable heatmaps, and (3) plots the hidden representations of vision and language tokens as they pass through the transformer layers. In this paper, we demonstrate the functionalities of VL-InterpreT through the analysis of KD-VLP, an end-to-end pretraining vision-language multimodal transformer-based model, in the tasks of Visual Commonsense Reasoning (VCR) and WebQA, two visual question answering benchmarks. Furthermore, we also present a few interesting findings about multimodal transformer behaviors that were learned through our tool.Keywords
This publication has 20 references indexed in Scilit:
- Learning With Interpretable Structure From Gated RNNIEEE Transactions on Neural Networks and Learning Systems, 2020
- Entangled Transformer for Image Captioning2019
- The What-If Tool: Interactive Probing of Machine Learning ModelsIEEE Transactions on Visualization and Computer Graphics, 2019
- Interpreting CNNs via Decision Trees2019
- Analyzing the Structure of Attention in a Transformer Language Model2019
- Open Sesame: Getting inside BERT’s Linguistic Knowledge2019
- What Does BERT Look at? An Analysis of BERT’s Attention2019
- Linguistic Knowledge and Transferability of Contextual Representations2019
- LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural NetworksIEEE Transactions on Visualization and Computer Graphics, 2017
- On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance PropagationPLOS ONE, 2015