Reinforcement Learning Based Robust Volt/Var Control in Active Distribution Networks With Imprecisely Known Delay
Preprint
Abstract
Active distribution networks (ADNs) incorporating massive photovoltaic (PV) devices encounter challenges of rapid voltage fluctuations and potential violations. Due to the fluctuation and intermittency of PV generation, the state gap, arising from time-inconsistent states and exacerbated by imprecisely known system delays, significantly impacts the accuracy of voltage control. This paper addresses this challenge by introducing a framework for delay adaptive Volt/Var control (VVC) in the presence of imprecisely known system delays to regulate the reactive power of PV inverters. The proposed approach formulates the voltage control, based on predicted system operation states, as a robust VVC problem. It employs sample selection from the state prediction interval to promptly identify the worst-performing system operation state. Furthermore, we leverage the decentralized partially observable Markov decision process (Dec-POMDP) to reformulate the robust VVC problem. We design Multiple Policy Networks and employ Multiple Policy Networks and Reward Shaping-based Multi-agent Twin Delayed Deep Deterministic Policy Gradient (MPNRS-MATD3) algorithm to efficiently address and solve the Dec-POMDP model-based problem. Simulation results show the delay adaption characteristic of our proposed framework, and the MPNRS-MATD3 outperforms other multi-agent reinforcement learning algorithms in robust voltage control.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Distributed Energy Resources Based Two-Layer Delay-Independent Voltage Coordinated Control in Active Distribution NetworkIEEE Transactions on Industrial Informatics, 2023
- Reward Shaping-Based Actor–Critic Deep Reinforcement Learning for Residential Energy ManagementIEEE Transactions on Industrial Informatics, 2022
- Regional Coordinated Voltage Regulation in Active Distribution Networks With PV-BESSIEEE Transactions on Circuits and Systems II: Express Briefs, 2022
- Distributed Voltage Regulation for Low-Voltage and High-PV-Penetration Networks With Battery Energy Storage Systems Subject to Communication DelayIEEE Transactions on Control Systems Technology, 2021
- DeepAR: Probabilistic forecasting with autoregressive recurrent networksInternational Journal of Forecasting, 2020
- A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement LearningIEEE Transactions on Power Systems, 2020
- Review of Challenges and Research Opportunities for Voltage Control in Smart GridsIEEE Transactions on Power Systems, 2019
- Quasi-Static Time-Series PV Hosting Capacity Methodology and Metrics2019
- A Concise Introduction to Decentralized POMDPs2016
- Options for Control of Reactive Power by Distributed Photovoltaic GeneratorsProceedings of the IEEE, 2011