Keeping Designers in the Loop
Authors:
Open Access
- 3 July 2020
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1245-1257
Abstract
Artificial intelligence algorithms have been used to enhance a wide variety of products and services, including assisting human decision making in high-stake contexts. However, these algorithms are complex and have trade-offs, notably between prediction accuracy and fairness to population subgroups. This makes it hard for designers to understand algorithms and design products or services in a way that respects users' goals, values, and needs. We proposed a method to help designers and users explore algorithms, visualize their trade-offs, and select algorithms with trade-offs consistent with their goals and needs. We evaluated our method on the problem of predicting criminal defendants' likelihood to re-offend through (i) a large-scale Amazon Mechanical Turk experiment, and (ii) in-depth interviews with domain experts. Our evaluations show that our method can help designers and users of these systems better understand and navigate algorithmic trade-offs. This paper contributes a new way of providing designers the ability to understand and control the outcomes of algorithmic systems they are creating.Keywords
Funding Information
- JP Morgan Faculty Award
- National Science Foundation (IIS-2001851,IIS-2000782,IIS-1939606)
This publication has 32 references indexed in Scilit:
- Interacting with Predictions2016
- International Perspectives on the Practical Application of Violence Risk Assessment: A Global Survey of 44 CountriesInternational Journal of Forensic Mental Health, 2014
- Identifying careless responses in survey data.Psychological Methods, 2012
- Measurement of information and communication technology experience and attitudes to e‐learning of students in the healthcare professions: integrative reviewJournal of Advanced Nursing, 2009
- The Recent Past and Near Future of Risk and/or Need AssessmentCrime & Delinquency, 2006
- Rapid Contextual DesignUbiquity, 2005
- Development and evaluation of emerging design patterns for ubiquitous computing2004
- On-line trust: concepts, evolving themes, a modelInternational Journal of Human-Computer Studies, 2003
- False Positives, False Negatives, and the Validity of the Diagnosis of Major Depression in Primary CareArchives of Family Medicine, 1998
- Trust, control strategies and allocation of function in human-machine systemsErgonomics, 1992