Compensating for Sensing Failures via Delegation in Human–AI Hybrid Systems
Open Access
Abstract
Given the increasing prevalence of intelligent systems capable of autonomous actions or augmenting human activities, it is important to consider scenarios in which the human, autonomous system, or both can exhibit failures as a result of one of several contributing factors (e.g., perception). Failures for either humans or autonomous agents can lead to simply a reduced performance level, or a failure can lead to something as severe as injury or death. For our topic, we consider the hybrid human–AI teaming case where a managing agent is tasked with identifying when to perform a delegated assignment and whether the human or autonomous system should gain control. In this context, the manager will estimate its best action based on the likelihood of either (human, autonomous) agent’s failure as a result of their sensing capabilities and possible deficiencies. We model how the environmental context can contribute to, or exacerbate, these sensing deficiencies. These contexts provide cases where the manager must learn to identify agents with capabilities that are suitable for decision-making. As such, we demonstrate how a reinforcement learning manager can correct the context–delegation association and assist the hybrid team of agents in outperforming the behavior of any agent working in isolation.Keywords
All Related Versions
Funding Information
- H2020 Humane-AI-Net (952026, CHIST-ERA-19-XAI-010-MUR-484-22, PE0000013)
- CHIST-ERA (952026, CHIST-ERA-19-XAI-010-MUR-484-22, PE0000013)
- European Union under the Italian National Recovery and Resilience Plan (NRRP) of partnership on “Artificial Intelligence: Foundational Aspects” (952026, CHIST-ERA-19-XAI-010-MUR-484-22, PE0000013)
This publication has 32 references indexed in Scilit:
- Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving2019
- Is it Safe to Drive? An Overview of Factors, Metrics, and Datasets for Driveability Assessment in Autonomous DrivingIEEE Transactions on Intelligent Transportation Systems, 2019
- Automated Evaluation of Semantic Segmentation Robustness for Autonomous DrivingIEEE Transactions on Intelligent Transportation Systems, 2019
- A Systematic Review of Perception System and Simulators for Autonomous Vehicles ResearchSensors, 2019
- Robust Physical-World Attacks on Deep Learning Visual Classification2018
- Failure Prediction for Autonomous Driving2018
- What We Can Learn from Pilots for Handovers and (De)Skilling in Semi-Autonomous Driving2017
- To delegate or not to delegate: A review of control frameworks for autonomous carsApplied Ergonomics, 2016
- Autonomous driving in urban environments: Boss and the Urban ChallengeJournal of Field Robotics, 2008
- Quality of service oriented spatial processing in the Manhattan grid2008