Selective Explanations: Leveraging Human Input to Align Explainable AI
Authors:
- Vivian LaiVLVivian Lai
- Yiming ZhangYZYiming Zhang
- Chacha ChenCCChacha Chen
- Q. Vera LiaoQLQ. Vera Liao
- Chenhao TanCTChenhao Tan
Preprint
Abstract
While a vast collection of explainable AI (XAI) algorithms have been developed in recent years, they are often criticized for significant gaps with how humans produce and consume explanations. As a result, current XAI techniques are often found to be hard to use and lack effectiveness. In this work, we attempt to close these gaps by making AI explanations selective -- a fundamental property of human explanations -- by selectively presenting a subset from a large set of model reasons based on what aligns with the recipient's preferences. We propose a general framework for generating selective explanations by leveraging human input on a small sample. This framework opens up a rich design space that accounts for different selectivity goals, types of input, and more. As a showcase, we use a decision-support task to explore selective explanations based on what the decision-maker would consider relevant to the decision task. We conducted two experimental studies to examine three out of a broader possible set of paradigms based on our proposed framework: in Study 1, we ask the participants to provide their own input to generate selective explanations, with either open-ended or critique-based input. In Study 2, we show participants selective explanations based on input from a panel of similar users (annotators). Our experiments demonstrate the promise of selective explanations in reducing over-reliance on AI and improving decision outcomes and subjective perceptions of the AI, but also paint a nuanced picture that attributes some of these positive effects to the opportunity to provide one's own input to augment AI explanations. Overall, our work proposes a novel XAI framework inspired by human communication behaviors and demonstrates its potentials to encourage future work to better align AI explanations with human production and consumption of explanations.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Explainability pitfalls: Beyond dark patterns in explainable AIPatterns, 2024
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AIInformation Fusion, 2020
- The Principles and Limits of Algorithm-in-the-Loop Decision MakingProceedings of the ACM on Human-Computer Interaction, 2019
- Stop explaining black box machine learning models for high stakes decisions and use interpretable models insteadNature Machine Intelligence, 2019
- Explanation in artificial intelligence: Insights from the social sciencesArtificial Intelligence, 2019
- Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)IEEE Access, 2018
- A Survey of Methods for Explaining Black Box ModelsACM Computing Surveys, 2018
- Interacting meaningfully with machine learning systems: Three experimentsInternational Journal of Human-Computer Studies, 2009
- For you can't always get what you want: When preconditions are better explanations than goalsBritish Journal of Social Psychology, 1997
- Varieties of knowledge elicitation techniquesInternational Journal of Human-Computer Studies, 1994