Reinforcement Learning-Based Energy Trading and Management of Regional Interconnected Microgrids
- 12 October 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Smart Grid
- Vol. 14 (3) , 2047-2059
Abstract
In this paper, we present a Value-Decomposition Deep Deterministic Policy Gradients (V3DPG) based Reinforcement Learning (RL) method for energy trading and management of regional interconnected microgrids (MGs). In practice, the state of an MG is time-varying and the traded energy flows continuously, which is generally neglected in researches. To address this problem, an Actor-Critic framework is adopted. Each MG has to make energy trading decision based on local observation and has no access to any knowledge of other MGs. We bring in the idea of value-decomposition in the training process to ensure the generation of feasible cooperative policies while maintaining MGs’ privacy and autonomous decision-making ability. Furthermore, in light of the uncertainty and fluctuation of renewable energy generation and users’ demand, a recurrent neural network (RNN) with Burn-In initialization is combined with critic network to achieve implicit predictions. Meanwhile, we also take Energy Storage System (ESS) with operational constraints into consideration and deem it as a virtual market innovatively. Experiments have been carried out under real-world data to verify the merit of the proposed method, compared to existing RL-based works.Keywords
Funding Information
- National Natural Science Foundation of China (61821004, 61922058, 62133008, 62173225)
This publication has 27 references indexed in Scilit:
- Modeling of Lithium-Ion Battery Degradation for Cell Life AssessmentIEEE Transactions on Smart Grid, 2016
- Distributed Energy Trading: The Multiple-Microgrid CaseIEEE Transactions on Industrial Electronics, 2014
- A Game-Theoretic Approach to Energy Trading in the Smart GridIEEE Transactions on Smart Grid, 2014
- Game-Theoretic Demand-Side Management With Storage Devices for the Future Smart GridIEEE Transactions on Smart Grid, 2014
- Residential Energy Consumption Scheduling: A Coupled-Constraint Game ApproachIEEE Transactions on Smart Grid, 2014
- Distributed Optimal Power Flow for Smart MicrogridsIEEE Transactions on Smart Grid, 2013
- An Adaptive $Q$-Learning Algorithm Developed for Agent-Based Computational Modeling of Electricity MarketIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2010
- Wind Power Generation2007
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Q-learningMachine Learning, 1992