HIVE: Evaluating the Human Interpretability of Visual Explanations
Preprint
Abstract
As AI technology is increasingly applied to high-impact, high-risk domains, there have been a number of new methods aimed at making AI models more human interpretable. Despite the recent growth of interpretability work, there is a lack of systematic evaluation of proposed techniques. In this work, we introduce HIVE (Human Interpretability of Visual Explanations), a novel human evaluation framework that assesses the utility of explanations to human users in AI-assisted decision making scenarios, and enables falsifiable hypothesis testing, cross-method comparison, and human-centered evaluation of visual interpretability methods. To the best of our knowledge, this is the first work of its kind. Using HIVE, we conduct IRB-approved human studies with nearly 1000 participants and evaluate four methods that represent the diversity of computer vision interpretability works: GradCAM, BagNet, ProtoPNet, and ProtoTree. Our results suggest that explanations engender human trust, even for incorrect predictions, yet are not distinct enough for users to distinguish between correct and incorrect predictions. We open-source HIVE to enable future studies and encourage more human-centered approaches to interpretability research.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- Making Heads or Tails: Towards Semantically Consistent Visual Counterfactuals2022
- Neural Networks Trained on Natural Scenes Exhibit Gestalt ClosureComputational Brain & Behavior, 2021
- Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach2020
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AIInformation Fusion, 2020
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based LocalizationInternational Journal of Computer Vision, 2019
- Interpretable Basis Decomposition for Visual Explanation2018
- Towards Realistic Predictors2018
- Top-Down Neural Attention by Excitation BackpropInternational Journal of Computer Vision, 2017
- Predicting Failures of Vision Systems2014
- The role of trust in automation relianceInternational Journal of Human-Computer Studies, 2003