Unified Automatic Control of Vehicular Systems With Reinforcement Learning
- 29 April 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 20 (2) , 789-804
Abstract
Emerging vehicular systems with increasing proportions of automated components present opportunities for optimal control to mitigate congestion and increase efficiency. There has been a recent interest in applying deep reinforcement learning (DRL) to these nonlinear dynamical systems for the automatic design of effective control strategies. Despite conceptual advantages of DRL being model-free, studies typically nonetheless rely on training setups that are painstakingly specialized to specific vehicular systems. This is a key challenge to efficient analysis of diverse vehicular and mobility systems. To this end, this article contributes a streamlined methodology for vehicular microsimulation and discovers high performance control strategies with minimal manual design. A variable-agent, multi-task approach is presented for optimization of vehicular Partially Observed Markov Decision Processes. The methodology is experimentally validated on mixed autonomy traffic systems, where fractions of vehicles are automated; empirical improvement, typically 15-60% over a human driving baseline, is observed in all configurations of six diverse open or closed traffic systems. The study reveals numerous emergent behaviors resembling wave mitigation, traffic signaling, and ramp metering. Finally, the emergent behaviors are analyzed to produce interpretable control strategies, which are validated against the learned control strategies.Keywords
All Related Versions
Funding Information
- Amazon Catalyst
- MIT-IBM Watson AI Laboratory
- Department of Transportation Dwight David Eisenhower Transportation Fellowship Program
This publication has 40 references indexed in Scilit:
- Help or hindrance? The travel, energy and carbon impacts of highly automated vehiclesTransportation Research Part A: Policy and Practice, 2016
- A Learning-Based Framework for Velocity Control in Autonomous DrivingIEEE Transactions on Automation Science and Engineering, 2015
- Ensemble Coordination Approach in Multi-AGV Systems Applied to Industrial WarehousesIEEE Transactions on Automation Science and Engineering, 2015
- Hysteresis and Capacity Drop Phenomena in Freeway NetworksTransportation Research Record: Journal of the Transportation Research Board, 2013
- Cooperative driving: an ant colony system for autonomous intersection managementApplied Intelligence, 2011
- Review of road traffic control strategiesProceedings of the IEEE, 2003
- Research advances in intelligent collision avoidance and adaptive cruise controlIEEE Transactions on Intelligent Transportation Systems, 2003
- Planning and acting in partially observable stochastic domainsArtificial Intelligence, 1998
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992
- Traffic Dynamics: Analysis of Stability in Car FollowingOperations Research, 1959