Realising Meaningful Human Control Over Automated Driving Systems: A Multidisciplinary Approach
Authors:
Open Access
- 28 July 2022
- journal article
- research article
- Published by Springer Nature in Minds and Machines
- Vol. 33 (4) , 587-611
Abstract
The paper presents a framework to realise “meaningful human control” over Automated Driving Systems. The framework is based on an original synthesis of the results of the multidisciplinary research project “Meaningful Human Control over Automated Driving Systems” lead by a team of engineers, philosophers, and psychologists at Delft University of the Technology from 2017 to 2021. Meaningful human control aims at protecting safety and reducing responsibility gaps. The framework is based on the core assumption that human persons and institutions, not hardware and software and their algorithms, should remain ultimately—though not necessarily directly—in control of, and thus morally responsible for, the potentially dangerous operation of driving in mixed traffic. We propose an Automated Driving System to be under meaningful human control if it behaves according to the relevant reasons of the relevant human actors (tracking), and that any potentially dangerous event can be related to a human actor (tracing). We operationalise the requirements for meaningful human control through multidisciplinary work in philosophy, behavioural psychology and traffic engineering. The tracking condition is operationalised via a proximal scale of reasons and the tracing condition via an evaluation cascade table. We review the implications and requirements for the behaviour and skills of human actors, in particular related to supervisory control and driver education. We show how the evaluation cascade table can be applied in concrete engineering use cases in combination with the definition of core components to expose deficiencies in traceability, thereby avoiding so-called responsibility gaps. Future research directions are proposed to expand the philosophical framework and use cases, supervisory control and driver education, real-world pilots and institutional embeddingKeywords
Funding Information
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (MVI.16.044)
This publication has 60 references indexed in Scilit:
- Developing a framework for responsible innovationResearch Policy, 2013
- Sensing and Actuation in Intelligent Vehicles2012
- Towards a dynamic balance between humans and automation: authority, ability, responsibility and control in shared and cooperative control situationsCognition, Technology & Work, 2011
- Saving lives in road traffic—ethical aspectsJournal of Public Health, 2009
- Killer RobotsJournal of Applied Philosophy, 2007
- Driving automation: learning from aviation about design philosophiesInternational Journal of Vehicle Design, 2007
- The responsibility gap: Ascribing responsibility for the actions of learning automataEthics and Information Technology, 2004
- Ironies of automationAutomatica, 1983
- Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings.The Journal of Abnormal and Social Psychology, 1963
- The Breakdown of Vigilance during Prolonged Visual SearchQuarterly Journal of Experimental Psychology, 1948