Deep Reinforcement Learning Lane-Changing Decision Algorithm for Intelligent Vehicles Combining LSTM Trajectory Prediction
Open Access
- 21 April 2024
- journal article
- research article
- Published by MDPI AG in World Electric Vehicle Journal
- Vol. 15 (4) , 173
Abstract
Intelligent decisions for autonomous lane-changing in vehicles have consistently been a focal point of research in the industry. Traditional lane-changing algorithms, which rely on predefined rules, are ill-suited for the complexities and variabilities of real-world road conditions. In this study, we propose an algorithm that leverages the deep deterministic policy gradient (DDPG) reinforcement learning, integrated with a long short-term memory (LSTM) trajectory prediction model, termed as LSTM-DDPG. In the proposed LSTM-DDPG model, the LSTM state module transforms the observed values from the observation module into a state representation, which then serves as a direct input to the DDPG actor network. Meanwhile, the LSTM prediction module translates the historical trajectory coordinates of nearby vehicles into a word-embedding vector via a fully connected layer, thus providing predicted trajectory information for surrounding vehicles. This integrated LSTM approach considers the potential influence of nearby vehicles on the lane-changing decisions of the subject vehicle. Furthermore, our study emphasizes the safety, efficiency, and comfort of the lane-changing process. Accordingly, we designed a reward and penalty function for the LSTM-DDPG algorithm and determined the optimal network structure parameters. The algorithm was then tested on a simulation platform built with MATLAB/Simulink. Our findings indicate that the LSTM-DDPG model offers a more realistic representation of traffic scenarios involving vehicle interactions. When compared to the traditional DDPG algorithm, the LSTM-DDPG achieved a 7.4% increase in average single-step rewards after normalization, underscoring its superior performance in enhancing lane-changing safety and efficiency. This research provides new ideas for advanced lane-changing decisions in autonomous vehicles.Keywords
Funding Information
- Hubei Province Key Research and Development Project (2023BAB169, 2022013702025184, 2022BGE248, BK202215)
- Major science and technology project in Wuhan, Hubei Province (2023BAB169, 2022013702025184, 2022BGE248, BK202215)
- central government guided special projects for local scientific and technological development (2023BAB169, 2022013702025184, 2022BGE248, BK202215)
- Hubei University of Automotive Technology PhD Fund Project (2023BAB169, 2022013702025184, 2022BGE248, BK202215)
This publication has 13 references indexed in Scilit:
- Autonomous Robot Racing Competitions: Truly Multivehicle Autonomous Racing Competitions [Competitions]IEEE Robotics & Automation Magazine, 2024
- Modeling and simulation of lane-changing and collision avoiding autonomous vehicles on superhighwaysPhysica A: Statistical Mechanics and its Applications, 2022
- Lane changing assistance strategy based on an improved probabilistic model of dynamic occupancy gridsFrontiers of Information Technology & Electronic Engineering, 2021
- Cooperative Driving of Automated Vehicles Using B-Splines for Trajectory PlanningIEEE Transactions on Intelligent Vehicles, 2021
- An improved A-Star based path planning algorithm for autonomous land vehiclesInternational Journal of Advanced Robotic Systems, 2020
- Decision-Making System for Lane Change Using Deep Reinforcement Learning in Connected and Automated DrivingElectronics, 2019
- CIRL: Controllable Imitative Reinforcement Learning for Vision-Based Self-driving2018
- Scenario Model Predictive Control for Lane Change Assistance and Autonomous Driving on HighwaysIEEE Intelligent Transportation Systems Magazine, 2017
- End-to-End Driving in a Realistic Racing Game with Deep Reinforcement Learning2017
- Longitudinal fuzzy control for autonomous overtaking2011