Control-Guided Reinforcement Learning for Cooperative Energy Management
- 19 June 2026
- conference paper
- Published by The CACHE Foundation, Inc.
- Vol. 6, 1558-1564
Abstract
Addressing the urgent transition to low-carbon energy systems requires microgrids capable of locally coordinating electricity generation, storage, and flexible consumption. Their efficient integration calls for control strategies that are scalable, privacy-preserving, and robust to uncertainty. To address such a challenging control problem, this work proposes a decentralised Multi-Agent Reinforcement Learning (MARL) approach based on the Cross-Entropy Method (CEM) for the coordination of prosumers, equipped with renewable generation and vehicle-to-grid capabilities. To improve sample efficiency and robustness, the policy is warm-started using Behaviour Cloning (BC) from a classical Proportional-Integral-Derivative (PID) controller, resulting in a hybrid BC–CEM framework. The proposed method is evaluated in a realistic microgrid simulation with stochastic demand and real weather and generation profiles. Results show that BC–CEM accelerates convergence and achieves lower energy costs compared to both PID control and randomly initialized CEM, without sacrificing comfort or mobility requirements. The findings highlight the effectiveness of combining derivative-free optimization with imitation learning in complex MARL tasks, such as energy flexibility coordination.Keywords
This publication has 12 references indexed in Scilit:
- Coordinated voltage control for distribution network and multi-microgrids based on improved EM-RACE multi-agent reinforcement learningInternational Journal of Electrical Power & Energy Systems, 2025
- A scalable demand-side energy management control strategy for large residential districts based on an attention-driven multi-agent DRL approachApplied Energy, 2025
- Control-Informed Reinforcement Learning for Chemical ProcessesIndustrial & Engineering Chemistry Research, 2025
- Comparative Analysis of Control Strategies for Microgrid Energy Management with a Focus on Reinforcement LearningIEEE Access, 2024
- Centralised rehearsal of decentralised cooperation: Multi-agent reinforcement learning for the scalable coordination of residential energy flexibilityApplied Energy, 2024
- Enhancing Grid-Interactive Buildings Demand Response: Sequential Update-Based Multiagent Deep Reinforcement Learning ApproachIEEE Internet of Things Journal, 2024
- Multi-agent deep reinforcement learning based distributed control architecture for interconnected multi-energy microgrid energy management and optimizationEnergy Conversion and Management, 2023
- Scalable multi-agent reinforcement learning for distributed control of residential energy flexibilityApplied Energy, 2022
- Building Energy Management With Reinforcement Learning and Model Predictive Control: A SurveyIEEE Access, 2022
- A Tutorial on the Cross-Entropy MethodAnnals of Operations Research, 2005