Machine learning and power relations
Author:
Open Access
- 25 February 2022
- journal article
- research article
- Published by Springer Nature in AI & SOCIETY
- Vol. 38 (4) , 1493-1500
Abstract
There has been an increased focus within the AI ethics literature on questions of power, reflected in the ideal of accountability supported by many Responsible AI guidelines. While this recent debate points towards the power asymmetry between those who shape AI systems and those affected by them, the literature lacks normative grounding and misses conceptual clarity on how these power dynamics take shape. In this paper, I develop a workable conceptualization of said power dynamics according to Cristiano Castelfranchi’s conceptual framework of power and argue that end-users depend on a system’s developers and users, because end-users rely on these systems to satisfy their goals, constituting a power asymmetry between developers, users and end-users. I ground my analysis in the neo-republican moral wrong of domination, drawing attention to legitimacy concerns of the power-dependence relation following from the current lack of accountability mechanisms. I illustrate my claims on the basis of a risk-prediction machine learning system, and propose institutional (external auditing) and project-specific solutions (increase contestability through design-for-values approaches) to mitigate domination.Keywords
This publication has 29 references indexed in Scilit:
- The ethics of algorithms: Mapping the debateBig Data & Society, 2016
- Algorithmic IF … THEN rules and the conditions and consequences of powerInformation, Communication & Society, 2016
- Value Sensitive Design: Applications, Adaptations, and Critiques2015
- Algorithmic AccountabilityDigital Journalism, 2014
- Routledge International Handbook of Participatory Design2012
- Rethinking the four dimensions of power: domination and empowermentJournal of Political Power, 2012
- The Instability of Freedom as Noninterference: The Case of Isaiah BerlinEthics, 2011
- A General Theory of Domination and Justice2010
- The responsibility gap: Ascribing responsibility for the actions of learning automataEthics and Information Technology, 2004
- The Micro-Macro Constitution of PowerProtoSociology, 2003