Bayesian Reinforcement Learning for Automatic Voltage Control under Cyber-Induced Uncertainty
Authors:
- Abhijeet SahuASAbhijeet Sahu
- Katherine DavisKDKatherine Davis
Preprint
Abstract
Voltage control is crucial to large-scale power system reliable operation, as timely reactive power support can help prevent widespread outages. However, there is currently no built in mechanism for power systems to ensure that the voltage control objective to maintain reliable operation will survive or sustain the uncertainty caused under adversary presence. Hence, this work introduces a Bayesian Reinforcement Learning (BRL) approach for power system control problems, with focus on sustained voltage control under uncertainty in a cyber-adversarial environment. This work proposes a data-driven BRL-based approach for automatic voltage control by formulating and solving a Partially-Observable Markov Decision Problem (POMDP), where the states are partially observable due to cyber intrusions. The techniques are evaluated on the WSCC and IEEE 14 bus systems. Additionally, BRL techniques assist in automatically finding a threshold for exploration and exploitation in various RL techniques.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future ChallengesIEEE Transactions on Smart Grid, 2022
- Graph Neural Networks Based Detection of Stealth False Data Injection Attacks in Smart GridsIEEE Systems Journal, 2021
- Multi-Source Multi-Domain Data Fusion for Cyberattack Detection in Power SystemsIEEE Access, 2021
- Adversarial Training for a Continuous Robustness Control Problem in Power Systems2021
- Design and evaluation of a cyber‐physical testbed for improving attack resilience of power systemsIET Cyber-Physical Systems: Theory & Applications, 2021
- A Learning-Based Solution for an Adversarial Repeated Game in Cyber–Physical Power SystemsIEEE Transactions on Neural Networks and Learning Systems, 2019
- Reward-based Monte Carlo-Bayesian reinforcement learning for cyber preventive maintenanceComputers & Industrial Engineering, 2018
- Variational Inference: A Review for StatisticiansJournal of the American Statistical Association, 2017
- Bayesian reinforcement learning in continuous POMDPs with gaussian processes2009
- State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and AlgorithmsManagement Science, 1982