Integrated Demand Response in Multi-Energy Microgrids: A Deep Reinforcement Learning-Based Approach
Open Access
Abstract
The increasing complexity of multi-energy coordinated microgrids presents a challenge for traditional demand response providers to adapt to end users’ multi-energy interactions. The primary aim of demand response providers is to maximize their total profits via designing a pricing strategy for end users. The main challenge lies in the fact that DRPs have no access to the end users’ private preferences. To address this challenge, we propose a deep reinforcement learning-based approach to devise a coordinated scheduling and pricing strategy without requiring any private information. First, we develop an integrated scheduling model that combines power and gas demand response by converting multiple energy sources with different types of residential end users. Then, we formulate the pricing strategy as a Markov Decision Process with an unknown transition. The novel soft actor-critic algorithm is utilized to efficiently train neural networks with the entropy function and to learn the pricing strategies to maximize demand response providers’ profits under various sources of uncertainties. Case studies are conducted to demonstrate the effectiveness of the proposed approach in both deterministic and stochastic environment settings. Our proposed approach is also shown to be effective in handling different levels of uncertainties and achieving the near-optimal pricing strategy.Keywords
This publication has 35 references indexed in Scilit:
- Game theory based interactive demand side management responding to dynamic pricing in price-based demand response of smart gridsApplied Energy, 2019
- From demand response to integrated demand response: review and prospect of research and applicationProtection and Control of Modern Power Systems, 2019
- Estimating the Profile of Incentive-Based Demand Response (IBDR) by Integrating Technical Models and Social-Behavioral FactorsIEEE Transactions on Smart Grid, 2019
- Deriving nonlinear models for incentive-based demand response programsInternational Journal of Electrical Power & Energy Systems, 2019
- Learning-Based Demand Response for Privacy-Preserving UsersIEEE Transactions on Industrial Informatics, 2019
- Reinforcement learning for demand response: A review of algorithms and modeling techniquesApplied Energy, 2019
- Flexibility in Multi-Energy Communities With Electrical and Thermal Storage: A Stochastic, Robust Approach for Multi-Service Demand ResponseIEEE Transactions on Smart Grid, 2017
- Review and prospect of integrated demand response in the multi-energy systemApplied Energy, 2017
- A Survey on Demand Response in Smart Grids: Mathematical Models and ApproachesIEEE Transactions on Industrial Informatics, 2015
- Human-level control through deep reinforcement learningNature, 2015