Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic
Open Access
- 6 May 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of Intelligent Transportation Systems
- Vol. 3, 397-410
Abstract
Building autonomous vehicles (AVs) is a complex problem, but enabling them to operate in the real world where they will be surrounded by human-driven vehicles (HVs) is extremely challenging. Prior works have shown the possibilities of creating inter-agent cooperation between a group of AVs that follow a social utility. Such altruistic AVs can form alliances and affect the behavior of HVs to achieve socially desirable outcomes. We identify two major challenges in the co-existence of AVs and HVs. First, social preferences and individual traits of a given human driver, e.g., selflessness and aggressiveness are unknown to an AV, and it is almost impossible to infer them in real-time during a short AV-HV interaction. Second, contrary to AVs that are expected to follow a policy, HVs do not necessarily follow a stationary policy and therefore are extremely hard to predict. To alleviate the above-mentioned challenges, we formulate the mixed-autonomy problem as a multi-agent reinforcement learning (MARL) problem and propose a decentralized framework and reward function for training cooperative AVs. Our approach enables AVs to learn the decision-making of HVs implicitly from experience, optimizes for a social utility while prioritizing safety and allowing adaptability; robustifying altruistic AVs to different human behaviors and constraining them to a safe action space. Finally, we investigate the robustness, safety and sensitivity of AVs to various HVs behavioral traits and present the settings in which the AVs can learn cooperative policies that are adaptable to different situations.Keywords
All Related Versions
Funding Information
- National Science Foundation (CNS-1932037, EPCN 1952920)
This publication has 35 references indexed in Scilit:
- Planning for Autonomous Cars that Leverage Effects on Human Actions2016
- A Review of Research on Driving Styles and Road SafetyHuman Factors: The Journal of the Human Factors and Ergonomics Society, 2015
- Learning driving styles for autonomous vehicles from demonstration2015
- The Prosocial and Aggressive Driving Inventory (PADI): A self-report measure of safe and unsafe driving behaviorsAccident Analysis & Prevention, 2014
- Intentionality, anger, coping, and ego defensiveness in reactive aggressive drivingJournal of Applied Social Psychology, 2014
- Distress Tolerance as a Predictor of Risky and Aggressive DrivingTraffic Injury Prevention, 2014
- Driving Style Analysis Using Data Mining TechniquesInternational Journal of Computers Communications & Control, 2010
- Reinforcement learning-based multi-agent system for network traffic signal controlIET Intelligent Transport Systems, 2010
- General Lane-Changing Model MOBIL for Car-Following ModelsTransportation Research Record: Journal of the Transportation Research Board, 2007
- Congested traffic states in empirical observations and microscopic simulationsPhysical Review E, 2000