Optimizing Mixed Autonomy Traffic Flow with Decentralized Autonomous Vehicles and Multi-Agent Reinforcement Learning
Open Access
- 19 April 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Cyber-Physical Systems
- Vol. 7 (2) , 1-22
Abstract
We study the ability of autonomous vehicles to improve the throughput of a bottleneck using a fully decentralized control scheme in a mixed autonomy setting. We consider the problem of improving the throughput of a scaled model of the San Francisco–Oakland Bay Bridge: a two-stage bottleneck where four lanes reduce to two and then reduce to one. Although there is extensive work examining variants of bottleneck control in a centralized setting, there is less study of the challenging multi-agent setting where the large number of interacting AVs leads to significant optimization difficulties for reinforcement learning methods. We apply multi-agent reinforcement algorithms to this problem and demonstrate that significant improvements in bottleneck throughput, from 20% at a 5% penetration rate to 33% at a 40% penetration rate, can be achieved. We compare our results to a hand-designed feedback controller and demonstrate that our results sharply outperform the feedback controller despite extensive tuning. Additionally, we demonstrate that the RL-based controllers adopt a robust strategy that works across penetration rates whereas the feedback controllers degrade immediately upon penetration rate variation. We investigate the feasibility of both action and observation decentralization and demonstrate that effective strategies are possible using purely local sensing. Finally, we open-source our code at https://github.com/eugenevinitsky/decentralized_bottlenecks .Keywords
All Related Versions
Funding Information
- National Science Foundation (CNS-1837244)
- AWS Machine Learning Research
- U.S. Department of Energy’s Office of Energy Efficiency and Renewable Energy (CID DE-EE0008872)
This publication has 7 references indexed in Scilit:
- Microscopic Traffic Simulation using SUMO2018
- SUMO’s Lane-Changing Model2015
- A new approach for combined freeway Variable Speed Limits and Coordinated Ramp Metering2010
- Toll Plaza Merging Traffic Control for Throughput MaximizationJournal of Transportation Engineering, 2010
- Relation between traffic density and capacity drop at three freeway bottlenecksTransportation Research Part B: Methodological, 2007
- Dynamic Network Simulation with AIMSUN2006
- Control of freeway traffic flow by variable speed signsTransportation Research Part B: Methodological, 1990