Cooperative Adaptive Cruise Control with Model-Based Imitation Reinforcement Learning in Mixed Traffic of Connected Autonomous and Human Driven Vehicles
- 30 March 2026
- journal article
- Published by SAGE Publications in Transportation Research Record: Journal of the Transportation Research Board
- Vol. 2680 (11) , 437-455
Abstract
Cooperative adaptive cruise control (CACC) can improve the traffic efficiency and safety of a platoon on the road. Most traditional CACC methods need to rely on accurate mathematical models, while those based on deep reinforcement learning (DRL) suffer from long training times and poor convergence. In this context, this study proposes a CACC framework based on imitation learning (IL) and DRL, which aims to improve the car-following efficiency and long platoon stability of connected autonomous vehicles (CAVs) in a mixed traffic environment. This method combines the optimization ability of model predictive control (MPC) and the adaptive learning characteristics of a soft actor-critic (SAC) algorithm. MPC is demonstrated as an expert, and the pre-training policy network is obtained by imitation learning. The pre-training network is then introduced into SAC’s actor network, which enhances training efficiency in the SAC algorithm. Numerical simulation results show that the improved DRL algorithm has better convergence in the training process. Compared with the baseline model, the proposed framework has higher reward, lower tracking error, and better platoon stability in the evaluation. In addition, the proposed model can efficiently complete the task of car-following under different penetration rates.Keywords
Funding Information
- National Key Research and Development Program of China (2023YFB2603500)
- Heilongjiang Postdoctoral Research Startup Fund Project (LBH-Q21172)
- Heilongjiang Provincial Universities (2023GJ05)
- Program for Young Talents of Basic Research in Universities of Heilongjiang Province (YQJH2023146)
This publication has 39 references indexed in Scilit:
- Improved SQP and SLSQP algorithms for feasible path-based process optimisationComputers & Chemical Engineering, 2024
- Model predictive control policy design, solutions, and stability analysis for longitudinal vehicle control considering shockwave dampingTransportation Research Part C: Emerging Technologies, 2023
- Longitudinal deep truck: Deep longitudinal model with application to sim2real deep reinforcement learning for heavy‐duty truck control in the fieldJournal of Field Robotics, 2022
- Reinforcement Learning based cooperative longitudinal control for reducing traffic oscillations and improving platoon stabilityTransportation Research Part C: Emerging Technologies, 2022
- Connected automated vehicle cooperative control with a deep reinforcement learning approach in a mixed traffic environmentTransportation Research Part C: Emerging Technologies, 2021
- Connected and Autonomous Vehicles and Infrastructures: A Literature ReviewInternational Journal of Pavement Research and Technology, 2021
- Data-driven optimal cooperative adaptive cruise control of heterogeneous vehicle platoons with unknown dynamicsScience China Information Sciences, 2020
- Distributed model predictive control approach for cooperative car-following with guaranteed local and string stabilityTransportation Research Part B: Methodological, 2019
- Robust local and string stability for a decentralized car following control strategy for connected automated vehiclesTransportation Research Part B: Methodological, 2019
- Cooperative platoon control for a mixed traffic flow including human drive vehicles and connected and autonomous vehiclesTransportation Research Part B: Methodological, 2018