Reinforcement Learning with Distributed MPC for Fuel-Efficient Platoon Control with Discrete Gear Transitions
Preprint
Abstract
Cooperative control of groups of autonomous vehicles (AVs), i.e., platoons, is a promising direction to improving the efficiency of autonomous transportation systems. In this context, distributed co-optimization of both vehicle speed and gear position can offer benefits for fuel-efficient driving. To this end, model predictive control (MPC) is a popular approach, optimizing the speed and gear-shift schedule while explicitly considering the vehicles' dynamics over a prediction window. However, optimization over both the vehicles' continuous dynamics and discrete gear positions is computationally intensive, and may require overly long sample times or high-end hardware for real-time implementation. This work proposes a reinforcement learning (RL)-based distributed MPC approach to address this issue. For each vehicle in the platoon, a policy is trained to select and fix the gear positions across the prediction window of a local MPC controller, leaving a significantly simpler continuous optimization problem to be solved as part of a distributed MPC scheme. In order to reduce the computational cost of training and facilitate the scalability of the proposed approach to large platoons, the policies are parameterized such that the emergent multi-agent RL problem can be decoupled into single-agent learning tasks. In addition, a recurrent neural-network (RNN) architecture is proposed for the gear selection policy, such that the learning is scalable even as the number of possible gear-shift schedules grows exponentially with the MPC prediction horizon. In highway-driving simulations, the proposed approach is shown to have a significantly lower computation burden and a comparable performance in terms of fuel-efficient platoon control, with respect to pure MPC-based co-optimization.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Learning-Based MPC for Fuel Efficient Control of Autonomous Vehicles With Discrete Gear SelectionIEEE Control Systems Letters, 2025
- Energy and mobility impacts of connected autonomous vehicles with co-optimization of speed and powertrain on mixed vehicle platoonsTransportation Research Part C: Emerging Technologies, 2022
- Constraint-Driven Optimal Control of Multiagent Systems: A Highway Platooning Case StudyIEEE Control Systems Letters, 2021
- On distributed model predictive control for vehicle platooning with a recursive feasibility guaranteeIFAC-PapersOnLine, 2017
- Distributed Model Predictive Control for Heterogeneous Vehicle Platoons Under Unidirectional TopologiesIEEE Transactions on Control Systems Technology, 2016
- A Learning-Based Framework for Velocity Control in Autonomous DrivingIEEE Transactions on Automation Science and Engineering, 2015
- Knitro: An Integrated Package for Nonlinear Optimization2006
- On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programmingMathematical Programming, 2005
- Systems with persistent disturbances: predictive control with restricted constraintsAutomatica, 2001
- Control of systems integrating logic, dynamics, and constraintsAutomatica, 1999