Interpretable Failure Detection with Human-Level Concepts
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (25) , 26326-26334
Abstract
Reliable failure detection holds paramount importance in safety-critical applications. Yet, neural networks are known to produce overconfident predictions for misclassified samples. As a result, it remains a problematic matter as existing confidence score functions rely on category-level signals, the logits, to detect failures. This research introduces an innovative strategy, leveraging human-level concepts for a dual purpose: to reliably detect when a model fails and to transparently interpret why. By integrating a nuanced array of signals for each category, our method enables a finer-grained assessment of the model's confidence. We present a simple yet highly effective approach based on the ordinal ranking of concept activation to the input image. Without bells and whistles, our method is able to significantly reduce the false positive rate across diverse real-world image classification benchmarks, specifically by 3.7% on ImageNet and 9.0% on EuroSAT.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Beyond Accuracy: Ensuring Correct Predictions With Correct Rationales2024
- Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification2023
- Rethinking Confidence Calibration for Failure Prediction2022
- Recent Advances in Open Set Recognition: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
- EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019
- Remote Sensing Image Scene Classification: Benchmark and State of the ArtProceedings of the IEEE, 2017