Combining Decision Making and Trajectory Planning for Lane Changing Using Deep Reinforcement Learning
- 3 March 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (9) , 16110-16136
Abstract
In the context of Automated Vehicles, the Automated Lane Change system, is fundamentally based upon the separate constructs of Perception, Decision making, Trajectory Planning, and Execution. However, in existing works there are many simplistic and unplausible assumptions in applying these constructs that severely restrict their operational effectiveness in realistic and complex driving scenarios. For instance, there are rigid assumptions about the disposition of vehicles and that lane-changing maneuvers can occur instantaneously, but that highly desirable features such as the ability for real-time trajectory re-planning are lacking. In this paper, we address these limitations through an integrated methodology for lane-change decision making and trajectory planning, in which a deep Reinforcement Learning algorithm with a safe action set technique is employed in decision making that is effectively coupled to a specially devised trajectory planning model. The proposed new methodology is computationally efficient, supporting real-time implementation, and provides for lane-changing maneuvers that can be made simultaneously with other vehicles and can be dynamically re-planned; thus, enabling flexible, robust, and safe lane-changing maneuvers under the guidance of a new decision-making module. Finally, the veracity of the proposed methodology in guiding a vehicle to improve travel times and accomplish high-level driving behaviors such as overtaking and desired-speed maintenance in a range of road traffic scenarios is demonstrated in a number of numerical experiments.Keywords
Funding Information
- National Natural Science Foundation of China (71971017, 91746201, 71621001)
This publication has 53 references indexed in Scilit:
- Reinforcement learning in robotics: A surveyThe International Journal of Robotics Research, 2013
- How to set the switches on this thingCurrent Opinion in Neurobiology, 2012
- Microscopic Traffic Flow Simulator VISSIM2010
- Integrated driving behavior modelingTransportation Research Part C: Emerging Technologies, 2007
- General Lane-Changing Model MOBIL for Car-Following ModelsTransportation Research Record: Journal of the Transportation Research Board, 2007
- Modelling vehicle interactions in microscopic simulation of merging and weavingTransportation Research Part C: Emerging Technologies, 2005
- Q-learningMachine Learning, 1992
- Introduction: The challenge of reinforcement learningMachine Learning, 1992
- A model for the structure of lane-changing decisionsTransportation Research Part B: Methodological, 1986
- Projection method for solving a singular system of linear equations and its applicationsNumerische Mathematik, 1971