Automatic generation control based on Exploration-Enhanced Contrastive Learning Optimal Adversary-Aware TD3 Algorithm
Authors:
- Lei XiLei Xi
- Hua TongHTHua Tong
- Kailun WeiKWKailun Wei
- Meng LiuMLMeng Liu
- Yu ShiYSYu Shi
Preprint
- 28 April 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
The large-scale integration of renewable energy sources introduces significant stochastic disturbances, posing substantial challenges to Automatic Generation Control (AGC) in modern power systems. Reinforcement learning (RL) has shown strong potential in addressing such stochastic optimization problems, particularly in continuous control scenarios. Among existing approaches, the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm has been widely adopted due to its effectiveness in continuous action spaces. However, TD3 often suffers from insufficient exploration and reduced robustness when operating in highly stochastic environments. To address these challenges, this paper proposes an exploration-enhanced contrastive learning and optimal adversary-aware TD3 algorithm (EECL-OATD3) for AGC that jointly improves exploration efficiency and control robustness within a single learning architecture. The proposed algorithm introduces a coupled learning mechanism, in which intrinsic exploration incentives are adaptively generated to guide the agent toward under-explored regions of the state space, while adversarial perturbation modeling is incorporated into the policy evaluation process to explicitly account for worst-case disturbances. The effectiveness of the proposed algorithm is validated on an improved IEEE standard two-area load frequency control (LFC) model and the southwest china three-area LFC model with a high proportion of renewable energy . Simulation results demonstrate that the proposed method achieves faster convergence, reduced frequency deviations, and improved AGC performance compared with existing RL-based approaches.Keywords
This publication has 11 references indexed in Scilit:
- Enhancing automatic generation control (AGC) for frequency stability in renewable-dominated power grids: Challenges, gaps, and future directionsHeliyon, 2025
- A bi-level optimization approach for siting and sizing of distributed wind-storage power systems in rural areas based on MCTS and IPSO algorithmsEnergy Conversion and Management, 2025
- Coordinated load management of building clusters and electric vehicles charging: An economic model predictive control investigation in demand responseEnergy Conversion and Management, 2025
- Enhancing grid stability through virtual inertia control in automatic generation control of a multi area systemEnergy Reports, 2025
- Prior Knowledge Incorporated Large-Scale Multiagent Deep Reinforcement Learning for Load Frequency Control of Isolated Microgrid Considering Multi-Structure CoordinationIEEE Transactions on Industrial Informatics, 2023
- Coordinated load frequency control of a smart hybrid power system using the DEMA-TD3 algorithmControl Engineering Practice, 2023
- Automatic Generation Control Strategy for Integrated Energy System Based on Ubiquitous Power Internet of ThingsIEEE Internet of Things Journal, 2022
- Wolf pack hunting strategy for automatic generation control of an islanding smart distribution networkEnergy Conversion and Management, 2016
- Advanced gas turbine cycles for power generation: a critical reviewApplied Thermal Engineering, 1998
- Multidimensional binary search trees used for associative searchingCommunications of the ACM, 1975