Bangji Fan

Service restoration implemented by multiple distributed energy resources (DERs) is a resilience-enhancing paradigm for modern distribution systems. To address the challenges of complex system modeling and the problem of cooperative control over heterogeneous multiple agents, this article proposes a graph reinforcement learning (G-RL) method based on heterogeneous multiagent systems (MASs). The method leverages graph-structured data to enhance the representation of distribution system states and employs graph attention networks (GATs) to deeply explore the power flow features and spatial characteristics of nodes in the restoration process. Additionally, a multihead self-attention (MHSA) is incorporated to strengthen collaboration among heterogeneous agents, enabling them to focus on relevant information from multiple perspectives during training. Finally, a joint simulation test platform is developed using Python and OpenDSS, and case studies on a 123-bus distribution system are conducted. Experimental results demonstrate that the proposed approach achieves efficient and autonomous service restoration by enhancing spatial feature extraction and improving collaborative decision-making among agents.

The large-scale integration of distributed energy resources into active distribution networks may significantly intensify voltage fluctuations and increase network losses. Traditional model-based reactive power optimization approaches depend on existence of accurate system models. On the other hand, conventional reinforcement learning methods largely ignore the spatial characteristics of the active distribution networks during training, allowing agents to have an inadequate perception of the system state. To address these challenges, this paper proposes a multi-agent deep reinforcement learning approach that integrates graph learning with reinforcement learning for the learning of reactive power optimization strategies in active distribution networks. Specifically, the active distribution network is divided into multiple regions, with each region being controlled by an agent. The agents collaborate to achieve the global reactive power optimization goal. The perception capability of the agents is enhanced by adopting graph attention networks during the feature extraction phase. In the training phase, a primal-dual method is employed to manage constraints effectively. During the execution phase, each agent controls the photovoltaic inverters, electric springs, and capacitor banks based on the strategies developed in the training phase. The performance of the proposed approach is validated by a series of experiments on the IEEE-33 system, along with comparisons versus some existing data-driven deep reinforcement learning methods.

Rail-based mobile energy storage (RMES) provides promising solutions to enhance power system resilience. This paper proposes an extended time-space network (TSN) model to characterize the impact of hurricanes on the rail-based transportation network (RTN). To coordinate the joint operation of the power transmission network and the RTN under a hurricane while considering the diffusional uncertainties about the random failures of the transmission lines and the transportation railway in both systems, a two-stage robust management scheme is presented to minimize the worst-case operating costs of the systems. Specifically, the first stage determines the pre-disaster on/off state of the generator and the pre-planned location of the RMES. In the second stage, the actual charging and discharging actions of RMES based on the proposed extended TSN model will be adjusted by the uncertainty realities to restore the load supply. To solve the developed scheduling model, a customized nested column-and-constraint generation (N-C&CG) algorithm is designed and validated on the IEEE reliability test system (RTS) with a 6-node railway network under hurricane. Case studies illustrate that the proposed RMES strategy can effectively improve power system resilience by exploiting mobility. Compared to classical approaches, the customized nested C&CG algorithm possesses strong sensing capability and computational efficiency.

The integration of distributed energy resources into distribution networks, marked by its inherent uncertainties, presents a substantial challenge for devising load restoration strategies. To tackle this challenge, we develop a memory-based graph reinforcement learning approach, designed to train the agent to acquire a critical load restoration strategy in a distribution network under uncertainties. Specifically, the restoration problem under uncertainties is formulated as a novel partially observable Markov decision process, where a multimodal feature-based observation space is proposed. This space includes graph-structured data of the environment and memory information of the agent. The graph-structured data contain potential features of the current observation, thus enhancing the observable domain, while the memory information incorporates temporal correlations between sample sequences to address the partial observability of the environment. Based on the proposed Markov process, we put forth a maximum entropy-based recurrent graph soft actor-critic algorithm to train the agent in partially observable environments through a recursive structure, where entropy regularization is utilized to facilitate a more extensive exploration of possibilities in a state space with high uncertainties. The performance of the proposed approach is validated by a comparative study versus existing results on the IEEE 123-bus system containing wind power and photovoltaic sources.

This paper proposes a joint strategy of pre-disaster hardening for transmission lines and real-time dispatching for units to address the failures with endogenous uncertainties caused by hurricanes on transmission lines. Specifically, we delve into the resilient unit commitment (UC) problem, which is formulated as a two-stage robust optimization (TRO) problem with endogenous uncertainty. In the first stage, the generator states, outputs, reserves, and line reinforcement are pre-scheduled to minimize defense costs. The second stage adopts a recourse approach to quantify the worst-case failure scenario of the lines, reflected as the costs of load shedding and generator curtailment. The operating status of transmission lines is depicted by an endogenous uncertainty set which is affected by the line hardening decision and unit reserve decision in the first stage. The TRO model with endogenous uncertainty is solved using a modified column-and-constraint generation (C&CG) algorithm. The proposed model is tested on two reliability test systems (RTS), including a modified IEEE RTS-79 and a two-area IEEE RTS-96 exposed to a hurricane. The results verify the effectiveness of the proposed method, compared with the popular two-stage robust optimization method with endogenous uncertainty.

With the advancement of artificial intelligence, deep reinforcement learning is emerging as an effective solution for distribution system service restoration. However, traditional deep reinforcement learning approaches are typically tailored for training agents in specific scenarios, limiting their ability to adapt rapidly to new environments. Furthermore, the spatial characteristics of the distribution network are largely ignored during the training, constraining the state perception capabilities of agents. To address these issues, this paper proposes a meta-based graph reinforcement learning approach that combines graph learning, meta-learning, and reinforcement learning for the learning of service restoration strategies in distribution network. The agent trained by such an approach possesses the feature perception capability of graph learning, allowing it to acquire deeper service restoration strategies from latent graph features. Moreover, the agent also has the fast adaptation ability of meta-learning, enabling it to quickly adapt to new restoration scenarios. Experimental results demonstrate that the proposed approach outperforms existing results of both specialized and generalized strategies.

With the ongoing integration of distributed energy resources, modern distribution systems are getting sufficient generation capacity to perform active restoration after outages without transmission system support. The model-based approaches are widely used in resolving service restoration problems, relying on accurate system models. Deep reinforcement learning is believed as an alternative solution for problem solving, although it has not been sufficiently explored. In this paper, the service restoration process is described as a partially observable Markov decision process, and a multi-agent graph reinforcement learning approach based on attention is proposed to train multiple agents to co-achieve the restoration goal to reinforce the system resilience in coping with extreme events. To consider the connections and correlations between nodes during the service restoration, the state of the active distribution network is defined by graph data that contains features of both topology and nodes. The perceived ability of the agents is empowered by graph convolutional networks during the feature extraction, supplying agents with more comprehensive data to learn more reasonable restoration strategies. In addition, the centralized training with attention is developed for multi-agent systems, focusing on the relations between the agents to strengthen teamwork capability. The performance of the proposed method is verified by a set of comparative studies on the IEEE-118 system with dispatchable generators, rooftop photovoltaics, and energy storage systems simultaneously.