Data Statements: From Technical Concept to Community Practice
Open Access
- 20 March 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Journal on Responsible Computing
- Vol. 1 (1) , 1-17
Abstract
Responsible computing ultimately requires that technical communities develop and adopt tools, processes, and practices that mitigate harms and support human flourishing. Prior efforts toward the responsible development and use of datasets, machine learning models, and other technical systems have led to the creation of documentation toolkits to facilitate transparency, diagnosis, and inclusion. This work takes the next step: to catalyze community uptake, alongside toolkit improvement. Specifically, starting from one such proposed toolkit specialized for language datasets, data statements for natural language processing, we explore how to improve the toolkit in three senses: (1) the content of the toolkit itself, (2) engagement with professional practice, and (3) moving from a conceptual proposal to a tested schema that the intended community of use may readily adopt. To achieve these goals, we first conducted a workshop with natural language processing practitioners to identify gaps and limitations of the toolkit as well as to develop best practices for writing data statements, yielding an interim improved toolkit. Then we conducted an analytic comparison between the interim toolkit and another documentation toolkit, datasheets for datasets. Based on these two integrated processes, we present our revised Version 2 schema and best practices in a guide for writing data statements. Our findings more generally provide integrated processes for co-evolving both technology and practice to address ethical concerns within situated technical communities.Keywords
Funding Information
- Tech Policy Lab
This publication has 20 references indexed in Scilit:
- Datasheets for datasetsCommunications of the ACM, 2021
- “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI2021
- Value sensitive design as a formative frameworkEthics and Information Technology, 2021
- FactSheets: Increasing trust in AI services through supplier's declarations of conformityIBM Journal of Research and Development, 2019
- Model Cards for Model Reporting2019
- Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better ScienceTransactions of the Association for Computational Linguistics, 2018
- Designing for energy transition through Value Sensitive DesignDesign Studies, 2018
- Discrimination in online ad deliveryCommunications of the ACM, 2013
- Development of a Privacy Addendum for Open Source Licenses: Value Sensitive Design in Industry2006
- Bias in computer systemsACM Transactions on Information Systems, 1996