Reinforcement Learning for Decision-Making and Control in Power Systems: Tutorial, Review, and Vision
Authors:
Preprint
Abstract
With large-scale integration of renewable generation and ubiquitous distributed energy resources (DERs), modern power systems confront a series of new challenges in operation and control, such as growing complexity, increasing uncertainty, and aggravating volatility. While the upside is that more and more data are available owing to the widely-deployed smart meters, smart sensors, and upgraded communication networks. As a result, data-driven control techniques, especially reinforcement learning (RL), have attracted surging attention in recent years. In this paper, we focus on RL and aim to provide a tutorial on various RL techniques and how they can be applied to the decision-making and control in power systems. In particular, we select three key applications, including frequency regulation, voltage control, and energy management, for illustration, and present the typical ways to model and tackle them with RL methods. We conclude by emphasizing two critical issues in the application of RL, i.e., safety and scalability. Several potential future directions are discussed as well.Keywords
All Related Versions
This publication has 76 references indexed in Scilit:
- Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms2021
- Load frequency regulation for multi‐area power system using integral reinforcement learningIET Generation, Transmission & Distribution, 2019
- (Deep) Reinforcement learning for electric power system control and related problems: A short review and perspectivesAnnual Reviews in Control, 2019
- A Reinforcement Learning Approach for Frequency Control of Inverted-Based MicrogridsIFAC-PapersOnLine, 2019
- Smart generation control based on multi-agent reinforcement learning with the idea of the time tunnelEnergy, 2018
- Artificial emotional reinforcement learning for automatic generation control of large‐scale interconnected power gridsIET Generation, Transmission & Distribution, 2017
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systemsAutomatica, 2014
- Batch Reinforcement Learning2012
- A reinforcement learning approach to automatic generation controlElectric Power Systems Research, 2002