POEM: Pattern-Oriented Explanations of Convolutional Neural Networks
Open Access
- 1 July 2023
- journal article
- conference paper
- Published by Association for Computing Machinery (ACM) in Proceedings of the VLDB Endowment
- Vol. 16 (11) , 3192-3200
Abstract
Convolutional Neural Networks (CNNs) are commonly used in computer vision. However, their predictions are difficult to explain, as is the case with many deep learning models. To address this problem, we present POEM, a modular framework that produces patterns of semantic concepts such as shapes and colours to explain image classifier CNNs. POEM identifies patterns such as "if sofa then living room", meaning that if an image contains a sofa and the model pays attention to the sofa, then the model classifies the image as a living room. We illustrate the advantages of POEM over existing work using quantitative and qualitative experiments.Keywords
This publication has 10 references indexed in Scilit:
- POEMProceedings of the VLDB Endowment, 2022
- Concept whitening for interpretable image recognitionNature Machine Intelligence, 2020
- Visualizing surrogate decision trees of convolutional neural networksJournal of Visualization, 2019
- Incremental and Approximate Inference for Faster Occlusion-based Deep CNN Explanations2019
- Interpreting CNNs via Decision Trees2019
- Unified Perceptual Parsing for Scene Understanding2018
- Net2Vec: Quantifying and Explaining How Concepts are Encoded by Filters in Deep Neural Networks2018
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization2017
- Interpretable Decision Sets2016
- Visualizing Deep Convolutional Neural Networks Using Natural Pre-imagesInternational Journal of Computer Vision, 2016