Participation in the age of foundation models
Open Access
- 3 June 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1609-1621
Abstract
Growing interest and investment in the capabilities of foundation models has positioned such systems to impact a wide array of services, from banking to healthcare. Alongside these opportunities is the risk that these systems reify existing power imbalances and cause disproportionate harm to historically marginalized groups. The larger scale and domain-agnostic manner in which these models operate further heightens the stakes: any errors or harms are liable to reoccur across use cases. In AI & ML more broadly, participatory approaches hold promise to lend agency and decision-making power to marginalized stakeholders, leading to systems that better benefit justice through equitable and distributed governance. But existing approaches in participatory AI/ML are typically grounded in a specific application and set of relevant stakeholders, and it is not straightforward how to apply these lessons to the context of foundation models. Our paper aims to fill this gap. First, we examine existing attempts at incorporating participation into foundation models. We highlight the tension between participation and scale, demonstrating that it is intractable for impacted communities to meaningfully shape a foundation model that is intended to be universally applicable. In response, we develop a blueprint for participatory foundation models that identifies more local, application-oriented opportunities for meaningful participation. In addition to the “foundation” layer, our framework proposes the “subfloor” layer, in which stakeholders develop shared technical infrastructure, norms and governance for a grounded domain such as clinical care, journalism, or finance, and the “surface” (or application) layer, in which affected communities shape the use of a foundation model for a specific downstream task. The intermediate “subfloor” layer scopes the range of potential harms to consider, and affords communities more concrete avenues for deliberation and intervention. At the same time, it avoids duplicative effort by scaling input across relevant use cases. Through three case studies in clinical care, financial services, and journalism, we illustrate how this multi-layer model can create more meaningful opportunities for participation than solely intervening at the foundation layer.Keywords
All Related Versions
This publication has 49 references indexed in Scilit:
- STELA: a community-centred approach to norm elicitation for AI alignmentScientific Reports, 2024
- Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation studyThe Lancet Digital Health, 2023
- The future landscape of large language models in medicineCommunications Medicine, 2023
- On Being Iterated: The Affective Demands of Design Participation2020
- Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design2020
- Toward situated interventions for algorithmic equity2020
- The Changing Contours of "Participation" in Data-driven, Algorithmic Ecosystems2018
- Needfinding: The Why and How of Uncovering People's NeedsDesign Management Journal (Former Series), 1999
- Participatory designCommunications of the ACM, 1993
- A Ladder Of Citizen ParticipationJournal of the American Institute of Planners, 1969