Distributional Deep Reinforcement Learning-Based Emergency Frequency Control
- 24 November 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Power Systems
- Vol. 37 (4) , 2720-2730
Abstract
Emergency frequency control is one of the most critical approaches to maintain power system stability after major disturbances. With the increasing number of grid-connected renewable energy sources, existing model-based methods of frequency control are facing up with challenges of computational speed and scalability for large-scale systems. In this paper, the emergency frequency control problem is formulated as a Markov Decision Process and solved through a novel distributional deep reinforcement learning (DRL) method, namely the distributional soft actor critic (DSAC) method. Compared with other reinforcement learning methods that only estimate the mean value, the proposed DSAC model estimates the distribution of value function over returns. This advancement can lead to more insights and knowledge for the agent, with the benefit of a much faster and more stable learning process, and the improved frequency control performance. The simulation results on IEEE 39-bus and IEEE 118-bus systems demonstrate the effectiveness and robustness of proposed models, as well as the advantage compared to other state-of-the-art DRL algorithms.Keywords
Funding Information
- U.S. Department of Energy’s Office of Energy Efficiency
- Renewable Energy
- Solar Energy Technology Office (DE-EE0009339)
This publication has 22 references indexed in Scilit:
- Semi-Adaptive Setting of Under Frequency Load Shedding Relays Considering Credible Generation Outage ScenariosIEEE Transactions on Power Delivery, 2018
- Robust Online Estimation of Power System Center of Inertia FrequencyIEEE Transactions on Power Systems, 2018
- Robust Frequency Divider for Power System Online Monitoring and ControlIEEE Transactions on Power Systems, 2017
- Optimization of the Event-Driven Emergency Load-Shedding Considering Transient Security and Stability ConstraintsIEEE Transactions on Power Systems, 2016
- WAMS-Based Underfrequency Load Shedding With Short-Term Frequency PredictionIEEE Transactions on Power Delivery, 2015
- Parallel‐differential evolution approach for optimal event‐driven load shedding against voltage collapse in power systemsIET Generation, Transmission & Distribution, 2014
- Real-time prediction of event-driven load shedding for frequency stability enhancement of power systemsIET Generation, Transmission & Distribution, 2012
- Adaptive Underfrequency Load Shedding Based on the Magnitude of the Disturbance EstimationIEEE Transactions on Power Systems, 2006
- A low-order system frequency response modelIEEE Transactions on Power Systems, 1990
- A Practical Method for the Direct Analysis of Transient StabilityIEEE Transactions on Power Apparatus and Systems, 1979