Ethical Redress of Racial Inequities in AI: Lessons from Decoupling Machine Learning from Optimization in Medical Appointment Scheduling
Authors:
Open Access
- 20 October 2022
- journal article
- research article
- Published by Springer Nature in Philosophy & Technology
- Vol. 35 (4) , 1-19
Abstract
An Artificial Intelligence algorithm trained on data that reflect racial biases may yield racially biased outputs, even if the algorithm on its own is unbiased. For example, algorithms used to schedule medical appointments in the USA predict that Black patients are at a higher risk of no-show than non-Black patients, though technically accurate given existing data that prediction results in Black patients being overwhelmingly scheduled in appointment slots that cause longer wait times than non-Black patients. This perpetuates racial inequity, in this case lesser access to medical care. This gives rise to one type of Accuracy-Fairness trade-off: preserve the efficiency offered by using AI to schedule appointments or discard that efficiency in order to avoid perpetuating ethno-racial disparities. Similar trade-offs arise in a range of AI applications including others in medicine, as well as in education, judicial systems, and public security, among others. This article presents a framework for addressing such trade-offs where Machine Learning and Optimization components of the algorithm are decoupled. Applied to medical appointment scheduling, our framework articulates four approaches intervening in different ways on different components of the algorithm. Each yields specific results, in one case preserving accuracy comparable to the current state-of-the-art while eliminating the disparity.Keywords
This publication has 58 references indexed in Scilit:
- FIFTY YEARS LATER: THE LEGACY OF THE CIVIL RIGHTS ACT OF 1964Journal of Policy Analysis and Management, 2015
- No-Show to Primary Care AppointmentsJournal of Primary Care & Community Health, 2013
- Appointment Scheduling with No‐Shows and OverbookingProduction and Operations Management, 2013
- Do wealth disparities contribute to health disparities within racial/ethnic groups?Journal of Epidemiology and Community Health, 2013
- A comparative analysis of machine learning techniques for student retention managementDecision Support Systems, 2010
- From Color-Blind to Post-Racial: Blacks and Social Justice in the Twenty-First CenturyJournal of Social Philosophy, 2010
- Racial/Ethnic Differences in Income Inequality across US RegionsThe Review of Black Political Economy, 2010
- Unpacking “Institutional Racism”: Insights from Wittgenstein, Garfinkel, Schutz, Goffman, and SacksSchutzian Research. A Yearbook of Worldly Phenomenology and Qualitative Social Science, 2010
- Understanding Unconscious Bias and Unintentional RacismPhi Delta Kappan, 2009
- The effect of income inequality on the health of selected US demographic groupsAmerican Journal of Public Health, 2000