A Novel Automatic Generation Control Method Based on the Large-Scale Electric Vehicles and Wind Power Integration Into the Grid
- 5 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (5) , 5824-5834
Abstract
In order to solve the problem of frequency instability of power system due to strong random disturbance caused by large-scale electric vehicles and wind power grid connection, an improved reinforcement learning algorithm, namely, optimistic initialized double Q, is proposed in this article from the perspective of automatic generation control. The proposed algorithm uses the optimistic initialization principle to expand the agent action exploration space, so as to prevent Q-learning from falling into local optimum by greedy strategy; meanwhile, it integrates double Q-learning to solve the problem of overestimation of action value in traditional reinforcement learning based on Q-learning. In the algorithm, the hyperparameter $\alpha_{\tau}$ is introduced to improve the learning efficiency, and the reward $b_{\tau}$ based on exploration times is introduced to increase the $Q$ value estimation to drive the exploration of the algorithm, so as to obtain the optimal solution. By simulating the two-area load frequency control model integrated with large-scale electric vehicles and the four-area interconnected power grid model integrated with large-scale wind power generation, it is verified that the proposed algorithm can obtain the global optimal solution, thus effectively solvinng the frequency instability caused by strong random disturbance in the grid-connected mode of large-scale wind power generation, and compared with many reinforcement learning algorithms, the proposed algorithm has better control performance.
Keywords
Funding Information
- National Natural Science Foundation of China (51707102)
This publication has 14 references indexed in Scilit:
- Coordinated control of gas supply system in PEMFC based on multi-agent deep reinforcement learningInternational Journal of Hydrogen Energy, 2021
- Two-Stage Full-Data Processing for Microgrid Planning With High Penetrations of Renewable Energy SourcesIEEE Transactions on Sustainable Energy, 2021
- Efficient experience replay based deep deterministic policy gradient for AGC dispatch in integrated energy systemApplied Energy, 2021
- A Multi-Step Unified Reinforcement Learning Method for Automatic Generation Control in Multi-Area Interconnected Power GridIEEE Transactions on Sustainable Energy, 2020
- Automatic Generation Control Based on Multiple Neural Networks With Actor-Critic StrategyIEEE Transactions on Neural Networks and Learning Systems, 2020
- Automatic generation control based on multiple-step greedy attribute and multiple-level allocation strategyCSEE Journal of Power and Energy Systems, 2020
- A Novel Multi-Agent DDQN-AD Method-Based Distributed Strategy for Automatic Generation Control of Integrated Energy SystemsIEEE Transactions on Sustainable Energy, 2019
- Smart generation control based on multi-agent reinforcement learning with the idea of the time tunnelEnergy, 2018
- Lifelong Learning for Complementary Generation Control of Interconnected Power Grids With High-Penetration Renewables and EVsIEEE Transactions on Power Systems, 2017
- Stochastic Optimal Relaxed Automatic Generation Control in Non-Markov Environment Based on Multi-Step $Q(\lambda)$ LearningIEEE Transactions on Power Systems, 2011