A Roadmap of Explainable Artificial Intelligence: Explain to Whom, When, What and How?
Open Access
- 24 November 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Autonomous and Adaptive Systems
- Vol. 19 (4) , 1-40
Abstract
Explainable artificial intelligence (XAI) has gained significant attention, especially in AI-powered autonomous and adaptive systems (AASs). However, a discernible disconnect exists among research efforts across different communities. The machine learning community often overlooks “explaining to whom,” while the human-computer interaction community has examined various stakeholders with diverse explanation needs without addressing which XAI methods meet these requirements. Currently, no clear guidance exists on which XAI methods suit which specific stakeholders and their distinct needs. This hinders the achievement of the goal of XAI: providing human users with understandable interpretations. To bridge this gap, this article presents a comprehensive XAI roadmap. Based on an extensive literature review, the roadmap summarizes different stakeholders, their explanation needs at different stages of the AI system lifecycle, the questions they may pose, and existing XAI methods. Then, by utilizing stakeholders’ inquiries as a conduit, the roadmap connects their needs to prevailing XAI methods, providing a guideline to assist researchers and practitioners to determine more easily which XAI methodologies can meet the specific needs of stakeholders in AASs. Finally, the roadmap discusses the limitations of existing XAI methods and outlines directions for future research.Keywords
Funding Information
- National Natural Science Foundation of China (62250710682)
- Guangdong Provincial Key Laboratory (2020B121201001)
- Program for Guangdong Introducing Innovative and Entrepreneurial Teams (2017ZT07X386)
This publication has 178 references indexed in Scilit:
- Harnessing eXplainable artificial intelligence for feature selection in time series energy forecasting: A comparative analysis of Grad-CAM and SHAPApplied Energy, 2023
- What do we want from Explainable Artificial Intelligence (XAI)? – A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI researchArtificial Intelligence, 2021
- Explainability as a non-functional requirement: challenges and recommendationsRequirements Engineering, 2020
- Explainability in human–agent systemsAutonomous Agents and Multi-Agent Systems, 2019
- Explanation in artificial intelligence: Insights from the social sciencesArtificial Intelligence, 2019
- Top-Down Neural Attention by Excitation BackpropInternational Journal of Computer Vision, 2017
- Auditing black-box models for indirect influenceKnowledge and Information Systems, 2017
- Fair, Transparent, and Accountable Algorithmic Decision-making ProcessesPhilosophy & Technology, 2017
- Financial credit risk assessment: a recent reviewArtificial Intelligence Review, 2015
- A peek into the black box: exploring classifiers by randomizationData Mining and Knowledge Discovery, 2014