Anti-Disturbance Self-Supervised Reinforcement Learning for Perturbed Car-Following System
- 26 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 72 (9) , 11318-11331
Abstract
This paper proposes an anti-disturbance car-following strategy for attenuating (i) exogenous disturbances from preceding traffic oscillations and (ii) endogenous disturbances in vehicular control systems (e.g., wind gust, ground friction, and rolling resistance). Firstly, it employs a modified robust controller to generate an expert car-following control experience. Subsequently, it imitates the expert behaviors via the behavioral cloning (BC) technique, thereby developing the anti-disturbance ability. Lastly, the obtained policy is optimized using the self-supervised reinforcement learning (RL) approach. The simulation experiments, comprising both training and evaluation phases, are performed via Python. To simulate car-following scenarios, we utilize the ground-truth data from the Next Generation Simulation (NGSIM) datasets. Through recursive interactions with the perturbed car-following environment, self-supervised RL drives stable policy improvement. The proposed anti-disturbance self-supervised RL (ADSSRL) policy presents a smooth and almost monotonously increasing reward curve. Further evaluation of disturbance dampening performance suggests that at least a 44.5% reduction in control efficiency cost and a 10.1% reduction in driving comfort cost are achieved compared with baselines.Keywords
Funding Information
- National Natural Science Foundation of China (52272331, 52232012)
- Scientific Research Foundation of Graduate School of Southeast University, China (YBPY2165)
This publication has 45 references indexed in Scilit:
- Hierarchical Design of Connected Cruise Control in the Presence of Information Delays and Uncertain Vehicle DynamicsIEEE Transactions on Control Systems Technology, 2017
- Robust control of heterogeneous vehicular platoon with uncertain dynamics and communication delayIET Intelligent Transport Systems, 2016
- Integrated optimal eco-driving on rolling terrain for hybrid electric vehicle with vehicle-infrastructure communicationTransportation Research Part C: Emerging Technologies, 2016
- Multiple-Model Switching Control of Vehicle Longitudinal Dynamics for Platoon-Level AutomationIEEE Transactions on Vehicular Technology, 2016
- Trajectory data reconstruction and simulation-based validation against macroscopic traffic patternsTransportation Research Part B: Methodological, 2015
- Learning Autonomous Driving Styles and Maneuvers from Expert Demonstration2013
- A behavioral car-following model that captures traffic oscillationsTransportation Research Part B: Methodological, 2012
- On the Necessity of Accurate IEEE 802.11P Models for IVC Protocol Simulation2012
- On the assessment of vehicle trajectory data accuracy and application to the Next Generation SIMulation (NGSIM) program dataTransportation Research Part C: Emerging Technologies, 2011
- Hierarchical platoon control with heterogeneous information feedbackIET Control Theory & Applications, 2011