On the Relationship Between Explanations, Fairness Perceptions, and Decisions
Preprint
Abstract
It is known that recommendations of AI-based systems can be incorrect or unfair. Hence, it is often proposed that a human be the final decision-maker. Prior work has argued that explanations are an essential pathway to help human decision-makers enhance decision quality and mitigate bias, i.e., facilitate human-AI complementarity. For these benefits to materialize, explanations should enable humans to appropriately rely on AI recommendations and override the algorithmic recommendation when necessary to increase distributive fairness of decisions. The literature, however, does not provide conclusive empirical evidence as to whether explanations enable such complementarity in practice. In this work, we (a) provide a conceptual framework to articulate the relationships between explanations, fairness perceptions, reliance, and distributive fairness, (b) apply it to understand (seemingly) contradictory research findings at the intersection of explanations and fairness, and (c) derive cohesive implications for the formulation of research questions and the design of experiments.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Explainability pitfalls: Beyond dark patterns in explainable AIPatterns, 2024
- Dimensions of Diversity in Human Perceptions of Algorithmic Fairness2022
- Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literatureBig Data & Society, 2022
- Fairness, explainability and in-between: understanding the impact of different explanation methods on non-expert users’ perceptions of fairness toward an algorithmic systemEthics and Information Technology, 2022
- Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision MakingProceedings of the ACM on Human-Computer Interaction, 2021
- What do we want from Explainable Artificial Intelligence (XAI)? – A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI researchArtificial Intelligence, 2021
- Fairness in Deep Learning: A Computational PerspectiveIEEE Intelligent Systems, 2020
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AIInformation Fusion, 2020
- The Principles and Limits of Algorithm-in-the-Loop Decision MakingProceedings of the ACM on Human-Computer Interaction, 2019
- Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction InstrumentsBig Data, 2017