XAIEV – A Framework for the Evaluation of XAI-Algorithms for Image Classification
Open Access
- 12 October 2025
- book chapter
- Published by Springer Nature
- p. 250-263
Abstract
Convolutional Neural Networks (CNNs), such as VGG and ResNet, have been widely used for image classification for several years. Numerous explainable AI (XAI) algorithms, including Grad-CAM and XRAI, have been proposed to enhance interpretability in this domain. However, a persistent challenge lies in quantitatively comparing different XAI algorithms, variants of the same algorithm, or combinations of CNN models and XAI algorithms. In this work, we introduce XAIEV – a versatile framework for computationally evaluating the quality of saliency-map-based XAI algorithms. This framework includes (A) a benchmark dataset (traffic sign recognition) with a known ground truth and (B) a software toolbox designed to facilitate the evaluation pipeline. The pipeline consists of four steps: (1) model training, (2) applying XAI algorithms to generate weighted saliency maps, (3) generating new test images with varying percentages of “important” pixels removed or retained, and (4) statistically evaluating accuracy changes on these test images and comparison to the ground truth. Based on this statistical evaluation, we define an Accuracy-Sensitivity Quotient (ASQ) as a novel quality metric for XAI algorithms applied to image classification. Using the XAIEV framework, we compare various combinations of CNN architectures (“SimpleCNN” (custom model), VGG, ResNet, ConvNext) with multiple XAI algorithms (Grad-CAM, XRAI, LIME, PRISM). Our numerical results reveal that the performance of XAI algorithms is highly dependent on the underlying CNN model.Keywords
This publication has 25 references indexed in Scilit:
- Interpretability is in the Eye of the Beholder: Human Versus Artificial Classification of Image Segments Generated by Humans Versus XAIInternational Journal of Human–Computer Interaction, 2024
- Building Ensemble of Deep Networks: Convolutional Networks and TransformersIEEE Access, 2023
- A systematic review of Explainable Artificial Intelligence models and applications: Recent developments and future trendsDecision Analytics Journal, 2023
- PRISM: Principal Image Sections Mapping2022
- CLEVR-XAI: A benchmark dataset for the ground truth evaluation of neural network explanationsInformation Fusion, 2022
- Enhancing Deep Neural Network Saliency Visualizations With Gradual ExtrapolationIEEE Access, 2021
- Learning Deep Features for Discriminative Localization2016
- Visualizing and Understanding Convolutional Networks2014
- ImageNet: A large-scale hierarchical image database2009
- Efficient Graph-Based Image SegmentationInternational Journal of Computer Vision, 2004