From Opacity to Transparency: User Behavior and Downstream Effects in Algorithmic Evaluation
- 15 June 2026
- journal article
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Information Systems Research
Abstract
Organizations are increasingly integrating artificial intelligence (AI) into traditionally human-driven evaluation processes, such as recruitment and performance evaluation. Yet the complexity of the underlying AI algorithms often renders these processes opaque, limiting users’ understanding and potentially inducing stress and other behavioral changes. In response to these challenges, transparency is often advocated as a mitigative approach, but its effects remain somewhat ambiguous. While transparency in algorithmic evaluation may mitigate interviewees’ stress in the recruitment context, it may also incentivize opportunistic impression management (IM), thereby engendering a transparency paradox. Focusing on this tension, we theorize that while algorithmic evaluation, relative to human evaluation, may heighten stress and deceptive IM in the recruitment process, transparency has the potential to mitigate these disruptions. We test this across eight studies, including two main experiments and a quasi-field replication. In Experiment I, we found that while algorithmic evaluation increased interviewees’ stress and deceptive IM relative to human evaluation, transparency counteracted these effects, aligning them closely with human evaluation. In Experiment II, we study how these effects caused deviations in downstream interview performance when such performance is assessed in human-only, AI-augmented (wherein human decision-makers are assisted by AI scores), or fully automated decision-making configurations. Although transparency consistently reduced deviations in stress and IM from human evaluation for interviewees, its corrective effect on interview performance was constrained on the evaluative side: when AI scores were present, evaluators anchored on them, discounting their own judgment, even as the AI scores failed to distinguish honest from deceptive behaviors. Together, these findings disentangle the dual-edged effects of transparency in algorithmic evaluation, showing that its benefits depend not only on how users respond, but also on how evaluation is conducted. This work advances research on algorithmic decision-making and socio-technical systems, and offers implications for organizations, policymakers, and AI developers.Keywords
This publication has 93 references indexed in Scilit:
- Practical challenges in mediation analysis: a guide for applied researchersHealth Services and Outcomes Research Methodology, 2024
- Shake and Fake: the Role of Interview Anxiety in Deceptive Impression ManagementJournal of Business and Psychology, 2020
- Explainable AI: from black box to glass boxJournal of the Academy of Marketing Science, 2019
- Machine learning approaches to facial and text analysis: Discovering CEO oral communication stylesStrategic Management Journal, 2019
- Explanation in artificial intelligence: Insights from the social sciencesArtificial Intelligence, 2019
- Behavioral Expression of Job Interview AnxietyJournal of Business and Psychology, 2015
- Secrets and Lies: Involuntary Leakage in Deceptive Facial Expressions as a Function of Emotional IntensityJournal of Nonverbal Behavior, 2011
- Persuade him by email, but see her in person: Online persuasion revisitedComputers in Human Behavior, 2007
- From flawed self‐assessment to blatant whoppers: the utility of voluntary and involuntary behavior in detecting deceptionBehavioral Sciences & the Law, 2006
- Social Behaviors as Determined by Different Arrangements of Social Consequences: Social Loafing, Social Facilitation, Deindividuation, and A Modified Social LoafingThe Psychological Record, 1999