Fairness in the Eye of the Beholder: How Gender Bias Alignment Influences Fairness Perceptions and Reliance in Human–AI Teams
Open Access
- 14 October 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Interactive Intelligent Systems
Abstract
Human-AI collaboration is increasingly relevant in consequential areas where AI recommendations support human discretion. However, human-AI teams’ effectiveness, capability, and fairness highly depend on human perceptions of AI. Positive fairness perceptions have been shown to foster trust and acceptance of AI recommendations. Yet, work on confirmation bias highlights that humans selectively adhere to AI recommendations that align with their expectations and beliefs—despite those recommendations not being necessarily correct or fair. This raises the question of whether confirmation bias also transfers to the alignment of gender bias between human and AI decisions. In our study, we examine how gender bias alignment and the presence of explanations accompanying the AI recommendation influence fairness perceptions, human self-confidence, and reliance on the AI system. Findings from two 2 \(\times\) 2 between-subjects studies underscore the relationship between gender-bias alignment, fairness perceptions, and reliance, demonstrating that merely constructing a “formally fair” AI system is insufficient for optimal human–AI collaboration. Ultimately, AI recommendations are likely to be overridden when the system's biases do not align with those of its users.
Keywords
This publication has 123 references indexed in Scilit:
- From human-centered to symbiotic artificial intelligence: a focus on medical applicationsMultimedia Tools and Applications, 2024
- Effective Human Oversight of AI-Based Systems: A Signal Detection Perspective on the Detection of Inaccurate and Unfair OutputsMinds and Machines, 2024
- On prediction-modelers and decision-makers: why fairness requires more than a fair prediction modelAI & SOCIETY, 2024
- Algorithmic discrimination in the credit domain: what do we know about it?AI & SOCIETY, 2023
- Gender bias perpetuation and mitigation in AI technologies: challenges and opportunitiesAI & SOCIETY, 2023
- Algorithmic fairness datasets: the story so farData Mining and Knowledge Discovery, 2022
- Fairness, explainability and in-between: understanding the impact of different explanation methods on non-expert users’ perceptions of fairness toward an algorithmic systemEthics and Information Technology, 2022
- Do the Ends Justify the Means? Variation in the Distributive and Procedural Fairness of Machine Learning AlgorithmsJournal of Business Ethics, 2021
- Explaining Recommendations: Design and Evaluation2015
- Narcissism, confidence, and risk attitudeJournal of Behavioral Decision Making, 2004