Human control of AI systems: from supervision to teaming
Top Cited Papers
Open Access
- 28 May 2024
- journal article
- research article
- Published by Springer Nature in AI and Ethics
- Vol. 5 (2) , 1535-1548
Abstract
This article reviews two main approaches to human control of AI systems: supervisory human control and human–machine teaming. It explores how each approach defines and guides the operational interplay between human behaviour and system behaviour to ensure that AI systems are effective throughout their deployment. Specifically, the article looks at how the two approaches differ in their conceptual and practical adequacy regarding the control of AI systems based on foundation models––i.e., models trained on vast datasets, exhibiting general capabilities, and producing non-deterministic behaviour. The article focuses on examples from the defence and security domain to highlight practical challenges in terms of human control of automation in general, and AI in particular, and concludes by arguing that approaches to human control are better served by an understanding of control as the product of collaborative agency in a multi-agent system rather than of exclusive human supervision.Keywords
Funding Information
- Economic and Social Research Council (R-DST-HCE / D034)
This publication has 56 references indexed in Scilit:
- Continuous Auditing of Artificial Intelligence: a Conceptualization and Assessment of Tools and FrameworksDigital Society, 2022
- A survey of human-in-the-loop for machine learningFuture Generation Computer Systems, 2022
- What Happens When Humans Believe Their Teammate is an AI? An Investigation into Humans Teaming with AutonomyComputers in Human Behavior, 2021
- HUMAN SUPERVISORY CONTROL OF AUTOMATION2021
- Accountability and Control Over Autonomous Weapon Systems: A Framework for Comprehensive Human OversightMinds and Machines, 2020
- Designing for Robust and Effective Teamwork in Human-Agent Teams2016
- Experience of agency and sense of responsibilityConsciousness and Cognition, 2011
- MABA-MABA or Abracadabra? Progress on Human-Automation Co-ordinationCognition, Technology & Work, 2002
- Understanding Perceived Complexity in Human Supervisory ControlCognition, Technology & Work, 2000
- Ironies of automationAutomatica, 1983