A Novel Energy Management Method for Networked Multi-Energy Microgrids Based on Improved DQN
- 27 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 14 (6) , 4912-4926
Abstract
The networked Multi-Energy Microgrids (MEMGs) can significantly improve the economic efficiency of operation through energy cooperation and complementarity. However, the complex energy trading mechanism and multi-energy coupling decision process present various challenges for the optimal operation of MEMGs, such as the large decision space and the difficulty of converging to the global optimal solution. This study presents a novel energy management method for MEMG based on an improved Deep Q-network (IDQN) to solve these problems. Firstly, the equivalent model of an external interactive environment is constructed for each MEMG by using a novel Kriging surrogate enhanced Gate Recurrent Unit-Temporal Convolutional Network (GRU-TCN), which requires only the accessible energy exchange data and the public and independent information as input. The computational complexity of the reinforcement learning reward function is reduced accordingly by using this equivalent model. Subsequently, an improved k-crossover sampling strategy with reduced access frequency of the low reward actions is proposed to replace the traditional.-greedy strategy to further improve the original DQN. This effectively addresses the problem of inefficient exploration in a large-scale action space. Lastly, the proposed method is verified using a test system with three MEMGs. The result demonstrates that the proposed IDQN method presents better convergence and stability than the traditional DQN, and presents better energy management and operational efficiency.Keywords
Funding Information
- National Natural Science Foundation of China (52177124, U2066211, 52107134)
- Youth Innovation Promotion Association of CAS (2019143)
- Institute of Electrical Engineering, CAS (E155610101)
- DNL Cooperation Fund, CAS (DNL202023)
This publication has 32 references indexed in Scilit:
- Microgrids energy management systems: A critical review on methods, solutions, and prospectsApplied Energy, 2018
- Game Theoretic Non-Cooperative Distributed Coordination Control for Multi-MicrogridsIEEE Transactions on Smart Grid, 2018
- Non‐cooperative and cooperative optimisation of battery energy storage system for energy management in multi‐microgridIET Generation, Transmission & Distribution, 2018
- Machine Health Monitoring Using Local Feature-Based Gated Recurrent Unit NetworksIEEE Transactions on Industrial Electronics, 2017
- Probabilistic optimal scheduling of networked microgrids considering time-based demand response programs under uncertaintyApplied Energy, 2017
- A Two-Layer Energy Management System for Microgrids With Hybrid Energy Storage Considering Degradation CostsIEEE Transactions on Smart Grid, 2017
- Distributed Energy Management for Networked Microgrids Using Online ADMM With RegretIEEE Transactions on Smart Grid, 2016
- Energy Management in Multi-Microgrid Systems — Development and AssessmentIEEE Transactions on Power Systems, 2016
- Coordinated Energy Management of Networked Microgrids in Distribution SystemsIEEE Transactions on Smart Grid, 2014
- Robust Energy Management for Microgrids With High-Penetration RenewablesIEEE Transactions on Sustainable Energy, 2013