A Robust Reinforcement Learning Framework for Platoon Control of Heterogeneous Vehicles Under Uncertain Dynamics
- 2 October 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (12) , 22031-22044
Abstract
Cooperative driving of connected and automated vehicles (CAVs) is envisioned as a promising approach to improving fuel efficiency, safety, and traffic flow. However, achieving robust and efficient control in heterogeneous CAV platoons remains challenging, especially under uncertain dynamics and external disturbances. Most existing reinforcement learning (RL)-based platoon controllers often ignore model uncertainties or rely on centralized training, limiting their scalability and robustness in real-world applicability. To address these limitations, this study proposes a fully distributed, model-free RL framework integrated with a robust compensator for optimal platoon control of heterogeneous vehicles with unknown dynamics. The RL agent simultaneously learns the optimal control policy and estimates control-relevant dynamic parameters using only local input-output data, without requiring explicit vehicle models. These estimates are then used in real time to construct a disturbance-rejection input that ensures robust trajectory tracking. A distributed observer based on consensus theory is embedded within the hybrid controller to estimate leader-relative reference trajectories using only local neighbor information, eliminating the need for global communication. Theoretical analysis guarantees policy convergence and bounded tracking errors under dynamic uncertainties. The proposed method is experimentally validated using the high-fidelity Mixed Traffic Simulation (MiTaS) platform, combining the SUMO microscopic traffic simulator with MATLAB, demonstrating improved tracking, damping of traffic oscillations, and up to 15.8% fuel savings compared to recent RL-based methods.Keywords
This publication has 39 references indexed in Scilit:
- Cooperative driving of heterogeneous uncertain nonlinear connected and autonomous vehicles via distributed switching robust PID-like controlInformation Sciences, 2023
- Robust learning‐based lateral tracking control for autonomous driving with input constraintsInternational Journal of Robust and Nonlinear Control, 2022
- Data‐driven resilient platooning control for vehicular platooning systems with denial‐of‐service attacksInternational Journal of Robust and Nonlinear Control, 2022
- Data-driven optimal cooperative adaptive cruise control of heterogeneous vehicle platoons with unknown dynamicsScience China Information Sciences, 2020
- Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous drivingTransportation Research Part C: Emerging Technologies, 2020
- Safety benefits of arterials’ crash risk under connected and automated vehiclesTransportation Research Part C: Emerging Technologies, 2019
- Human-like autonomous car-following model with deep reinforcement learningTransportation Research Part C: Emerging Technologies, 2018
- Adaptive multi-agents synchronization for collaborative driving of autonomous vehicles with multiple communication delaysTransportation Research Part C: Emerging Technologies, 2018
- Robust cooperation of connected vehicle systems with eigenvalue-bounded interaction topologies in the presence of uncertain dynamicsFrontiers of Mechanical Engineering, 2017
- Computationally efficient simultaneous policy update algorithm for nonlinear H∞ state feedback control with Galerkin's methodInternational Journal of Robust and Nonlinear Control, 2012