The Problem with Metrics is a Fundamental Problem for AI
Preprint
Abstract
Optimizing a given metric is a central aspect of most current AI approaches, yet overemphasizing metrics leads to manipulation, gaming, a myopic focus on short-term goals, and other unexpected negative consequences. This poses a fundamental contradiction for AI development. Through a series of real-world case studies, we look at various aspects of where metrics go wrong in practice and aspects of how our online environment and current business practices are exacerbating these failures. Finally, we propose a framework towards mitigating the harms caused by overemphasis of metrics within AI by: (1) using a slate of metrics to get a fuller and more nuanced picture, (2) combining metrics with qualitative accounts, and (3) involving a range of stakeholders, including those who will be most impacted.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Datasheets for datasetsCommunications of the ACM, 2021
- Auditing radicalization pathways on YouTube2020
- Dark Patterns at ScaleProceedings of the ACM on Human-Computer Interaction, 2019
- Toward inclusive tech policy design: a method for underrepresented voices to strengthen tech policy documentsEthics and Information Technology, 2019
- Does Machine Learning Automate Moral Hazard and Error?American Economic Review, 2017
- WHAT’S MEASURED IS WHAT MATTERS: TARGETS AND GAMING IN THE ENGLISH PUBLIC HEALTH CARE SYSTEMPublic Administration, 2006