Locating Risk: Task Designers and the Challenge of Risk Disclosure in Crowdsourced RAI Content Work CSCW029
Open Access
- 30 April 2026
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Human-Computer Interaction
- Vol. 10 (2) , 1-32
Abstract
As AI systems are increasingly tested and deployed in open-ended and high-stakes domains, crowdworkers are often tasked with responsible AI (RAI) content work. These tasks include labeling violent content, moderating disturbing text, or simulating harmful behavior for red teaming exercises to shape AI system behaviors. While prior research efforts have highlighted the risks to worker well-being associated with RAI content work, far less attention has been paid to how these risks are communicated to workers by task designers or individuals who design and post RAI tasks. Existing transparency frameworks and guidelines, such as model cards, datasheets, and crowdworksheets, focus on documenting model information and dataset collection processes, but they overlook an important aspect of disclosing well-being risks to workers. In the absence of standard workflows or clear guidance, the consistent application of content warnings, consent flows, or other forms of well-being risk disclosure remains unclear. This study investigates how task designers approach risk disclosure in crowdsourced RAI tasks. Drawing on interviews with 23 task designers across academic and industry sectors, we examine how well-being risk is recognized, interpreted, and communicated in practice. Our findings highlight the need to support task designers in identifying and communicating risks not only to support crowdworker well-being but also to strengthen the ethical integrity and technical efficacy of AI development pipelines.Keywords
Funding Information
- National Science Foundation (DGE2140739)
This publication has 86 references indexed in Scilit:
- Trustworthy human computation: a surveyArtificial Intelligence Review, 2024
- Cultivating Data Practices Across Boundaries: How Organizations Become Data-drivenComputer Supported Cooperative Work (CSCW), 2024
- The ethics of AI business practices: a review of 47 AI ethics guidelinesAI and Ethics, 2022
- Privacy in Crowdsourcing: a Review of the Threats and ChallengesComputer Supported Cooperative Work (CSCW), 2020
- Platformic Management, Boundary Resources for Gig Work, and Worker AutonomyComputer Supported Cooperative Work (CSCW), 2019
- Governing online platforms: From contested to cooperative responsibilityThe Information Society, 2017
- Unfairness by Design? The Perceived Fairness of Digital Labor on Crowdworking PlatformsJournal of Business Ethics, 2017
- Psychological treatments for adults with posttraumatic stress disorder: A systematic review and meta-analysisClinical Psychology Review, 2016
- Capture of Attention to Threatening Stimuli without Perceptual AwarenessCurrent Biology, 2009
- Predictors of amygdala activation during the processing of emotional stimuli: A meta-analysis of 385 PET and fMRI studiesBrain Research Reviews, 2008