Deep Reinforcement Learning for Energy Management in Hybrid Electric Vehicles With Softmax Double-Actor Regularized Critics
Open Access
- 3 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of Vehicular Technology
- Vol. 7, 723-736
Abstract
Enhancing fuel efficiency in hybrid electric vehicles (HEVs) requires energy management strategies (EMSs) that can operate effectively under nonlinear powertrain dynamics and uncertain, time-varying driving conditions. This paper proposes a deep reinforcement learning (DRL)- based EMS using the double actors regularized critics softmax deep deterministic policy gradient (DARC SD3) algorithm, which integrates Boltzmann-softmax value estimation, a dual-actor architecture, and critic regularization to improve learning stability and value-estimation accuracy. Simulation results show that the proposed DARC SD3 achieves faster convergence, improved state-of-charge (SOC) regulation, and reduced value estimation bias compared with DDPG, TD3, and baseline SD3. Under the FTP-75 driving cycle, the proposed EMS attains 94.6% of the dynamic programming (DP) benchmark fuel economy, while reducing engine transients and smoothing battery power flow. Further evaluation on an unseen composite driving cycle confirms that the trained policy maintains consistent fuel economy and SOC control, demonstrating strong generalization capability across diverse driving conditions.Keywords
This publication has 44 references indexed in Scilit:
- Optimization of multi-vehicle charging and discharging efficiency under time constraints based on reinforcement learningAlexandria Engineering Journal, 2024
- Accelerated Primal–Dual Deep Reinforcement Learning for Efficient Energy Management of Hybrid Electric VehiclesArabian Journal for Science and Engineering, 2024
- Deep reinforcement learning based optimization of automated guided vehicle time and energy consumption in a container terminalAlexandria Engineering Journal, 2023
- A Double‐Deep Q‐Network‐Based Energy Management Strategy for Hybrid Electric Vehicles under Variable Driving CyclesEnergy Technology, 2020
- Intelligent energy management for hybrid electric tracked vehicles using online reinforcement learningApplied Energy, 2019
- Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric busApplied Energy, 2019
- Adaptive Energy Management Strategy for Plug-in Hybrid Electric Vehicles with Pontryagin’s Minimum Principle Based on Daily Driving PatternsInternational Journal of Precision Engineering and Manufacturing-Green Technology, 2019
- Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicleApplied Energy, 2018
- Multi-objective component sizing based on optimal energy management strategy of fuel cell electric vehiclesApplied Energy, 2015
- A multi-level energy management system for multi-source electric vehicles – An integrated rule-based meta-heuristic approachApplied Energy, 2013