Secondary Frequency Control of Microgrids: An Online Reinforcement Learning Approach
- 25 March 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 67 (9) , 4824-4831
Abstract
In this article, we present a reinforcement learning-based scheme for secondary frequency control of lossy inverter-based microgrids. Compared with the existing methods in the literature, we relax the common restrictions on the system, i.e., being lossless, and the transmission lines and loads to have known constant impedances. The proposed secondary frequency control scheme does not require a priori information about system parameters and can achieve frequency synchronization within an ultimate bound in the presence of dominantly resistive and/or inductive line and load impedances, model parameter uncertainties, and time varying loads and disturbances. First, using Lyapunov theory, a feedback control is formulated based on the unknown dynamics of the microgrid. Next, a performance function is defined based on cumulative costs toward achieving convergence to the nominal frequency. The performance function is approximated by a critic neural network in real-time. An actor network is then simultaneously learning a parameterized approximation of the nonlinear dynamics and optimizing the approximated performance function obtained from the critic network. Furthermore, using the Lyapunov approach, the uniformly ultimate boundedness of the closed-loop frequency error dynamics and the networks' weight estimation errors are shown.Keywords
Funding Information
- Netherlands Organization for Scientific Research
- Uncertainty Reduction in Smart Energy Systems
This publication has 22 references indexed in Scilit:
- Bregman Storage Functions for Microgrid ControlIEEE Transactions on Automatic Control, 2017
- Gather-and-broadcast frequency control in power systemsAutomatica, 2017
- A survey on modeling of microgrids—From fundamental physics to phasors and voltage sourcesAutomatica, 2016
- A Lyapunov approach to control of microgrids with a network-preserved differential-algebraic model2016
- Secondary Frequency and Voltage Control of Islanded Microgrids via Distributed AveragingIEEE Transactions on Industrial Electronics, 2015
- Conditions for stability of droop-controlled inverter-based microgridsAutomatica, 2014
- Synchronization of droop-controlled microgrids with distributed rotational and electronic generation2013
- Reinforcement Learning-Based Output Feedback Control of Nonlinear Systems With Input ConstraintsIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2005
- Learning capability and storage capacity of two-hidden-layer feedforward networksIEEE Transactions on Neural Networks, 2003
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998