Deep reinforcement learning based voltage control revisited
Open Access
- 12 October 2023
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in IET Generation, Transmission & Distribution
- Vol. 17 (21) , 4826-4835
Abstract
Deep Reinforcement Learning (DRL) has shown promise for voltage control in power systems due to its speed and model‐free nature. However, learning optimal control policies through trial and error on a real grid is infeasible due to the mission‐critical nature of power systems. Instead, DRL agents are typically trained on a simulator, which may not accurately represent the real grid. This discrepancy can lead to suboptimal control policies and raises concerns for power system operators. In this paper, we revisit the problem of RL‐based voltage control and investigate how model inaccuracies affect the performance of the DRL agent. Extensive numerical experiments are conducted to quantify the impact of model inaccuracies on learning outcomes. Specifically, techniques that enable the DRL agent are focused on learning robust policies that can still perform well in the presence of model errors. Furthermore, the impact of the agent's decisions on the overall system loss are analyzed to provide additional insight into the control problem. This work aims to address the concerns of power system operators and make DRL‐based voltage control more practical and reliable.Keywords
Funding Information
- National Science Foundation (2301938)
- New Mexico Consortium (037211)
This publication has 26 references indexed in Scilit:
- Resilient Operation of Distribution Grids Using Deep Reinforcement LearningIEEE Transactions on Industrial Informatics, 2021
- AI-Based Autonomous Line Flow Control via Topology Adjustment for Maximizing Time-Series ATCs2020
- Planning and Execution using Inaccurate Models with Provable Guarantees2020
- A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement LearningIEEE Transactions on Power Systems, 2020
- Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid OperationsIEEE Transactions on Power Systems, 2019
- Adaptive Power System Emergency Control Using Deep Reinforcement LearningIEEE Transactions on Smart Grid, 2019
- Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning2019
- Reinforcement Learning for Cyber-Physical Security Assessment of Power Systems2019
- Data-Driven Load Frequency Control for Stochastic Power Systems: A Deep Reinforcement Learning Method With Continuous Action SearchIEEE Transactions on Power Systems, 2018
- Using inaccurate models in reinforcement learning2006