Selective Explanations: Leveraging Human Input to Align Explainable AI
Open Access
- 28 September 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 7 (CSCW2) , 1-35
Abstract
While a vast collection of explainable AI (XAI) algorithms has been developed in recent years, they have been criticized for significant gaps with how humans produce and consume explanations. As a result, current XAI techniques are often found to be hard to use and lack effectiveness. In this work, we attempt to close these gaps by making AI explanations selective ---a fundamental property of human explanations---by selectively presenting a subset of model reasoning based on what aligns with the recipient's preferences. We propose a general framework for generating selective explanations by leveraging human input on a small dataset. This framework opens up a rich design space that accounts for different selectivity goals, types of input, and more. As a showcase, we use a decision-support task to explore selective explanations based on what the decision-maker would consider relevant to the decision task. We conducted two experimental studies to examine three paradigms based on our proposed framework: in Study 1, we ask the participants to provide critique-based or open-ended input to generate selective explanations (self-input). In Study 2, we show the participants selective explanations based on input from a panel of similar users (annotator input). Our experiments demonstrate the promise of selective explanations in reducing over-reliance on AI and improving collaborative decision making and subjective perceptions of the AI system, but also paint a nuanced picture that attributes some of these positive effects to the opportunity to provide one's own input to augment AI explanations. Overall, our work proposes a novel XAI framework inspired by human communication behaviors and demonstrates its potential to encourage future work to make AI explanations more human-compatible.Keywords
All Related Versions
Funding Information
- National Science Foundation (IIS-2126602)
This publication has 69 references indexed in Scilit:
- "Why Should I Trust You?"2016
- Interpretable Decision Sets2016
- The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems2015
- Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology: General, 2015
- Interacting meaningfully with machine learning systems: Three experimentsInternational Journal of Human-Computer Studies, 2009
- Why and why not explanations improve the intelligibility of context-aware intelligent systems2009
- Opinion Mining and Sentiment AnalysisFoundations and Trends® in Information Retrieval, 2008
- For you can't always get what you want: When preconditions are better explanations than goalsBritish Journal of Social Psychology, 1997
- Varieties of knowledge elicitation techniquesInternational Journal of Human-Computer Studies, 1994
- Contrastive ExplanationRoyal Institute of Philosophy Supplement, 1990