RDRuisheng Diao

Ruisheng Diao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Ruisheng Diao has a total of 30 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Rethink AI-based Power Grid Control: Diving Into Algorithm Design

    by Xiren Zhou, Siqi Wang, Ruisheng Diao, Desong Bian, Jiahui Duan, Di Shi

    Recently, deep reinforcement learning (DRL)-based approach has shown promisein solving complex decision and control problems in power engineering domain.In this paper, we present an in-depth analysis of DRL-based voltage control fromaspects of algorithm selection, state space representation, and reward engineering.To resolve observed issues, we propose a novel imitation learning-based approachto directly map power grid operating points to effective actions without any interimreinforcement learning process. The performance results demonstrate that theproposed approach has strong generalization ability with much less training time.The agent trained by imitation learning is effective and robust to solve voltagecontrol problem and outperforms the former RL agents.

  • On Multi-Event Co-Calibration of Dynamic Model Parameters Using Soft Actor-Critic

    by Siqi Wang, Ruisheng Diao, Chunlei Xu, Di Shi, Zhiwei Wang

    Maintaining good quality of transient stability models for power system planning and operational analysis is of great importance. Identification and calibration of bad parameters using PMU measurements that work well for multiple events remains a challenging problem. In this letter, we present a novel parameter calibration method based on off-policy deep reinforcement learning (DRL) algorithm with maximum entropy, soft actor critic (SAC), to automatically tune incorrect parameter sets considering multiple events simultaneously, which can save tremendous labor efforts for maintaining model accuracy and complying with industry standards. The effectiveness of the proposed approach is verified through numerical experiments conducted on a realistic power plant model.


  • Imitation and Transfer Q-Learning-Based Parameter Identification for Composite Load Modeling

    by Jian Xie, Zixiao Ma, Kaveh Dehghanpour, Zhaoyu Wang, Yishen Wang, Ruisheng Diao, Di Shi

    Fast and accurate load parameter identification has a large impact on power systems operation and stability analysis. This paper proposes a novel Imitation and Transfer Q-learning (ITQ)-based method to identify parameters of composite constant impedance-current-power (ZIP) and induction motor (IM) load models. Firstly, an imitation learning process is introduced to improve the exploitation and exploration processes. Then, a transfer learning method is employed to overcome the challenge of time-consuming optimization when dealing with new identification tasks. An associative memory is designed to realize dimension reduction, knowledge learning and transfer between different identification tasks. Agents can exploit the optimal knowledge from source tasks to accelerate the search rate in new tasks and improve solution accuracy. A greedy action selection rule is adopted for agents to balance the global and local search. The performance of the proposed ITQ approach has been validated on a 68-bus test system. Simulation results in multi-test cases verify that the proposed method is robust and can estimate load parameters accurately. Comparisons with other methods show that the proposed method has superior convergence rate and stability.


  • Top Cited Papers

    A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement Learning

    by Shengyi Wang, Jiajun Duan, Di Shi, Chunlei Xu, Haifeng Li, Ruisheng Diao, Zhiwei Wang

    The complexity of modern power grids keeps increasing due to the expansion of renewable energy resources and the requirement of fast demand responses, which results in a great challenge for conventional power grid control systems. Existing autonomous control approaches for the power grid requires an accurate system model and a powerful computational platform, which is difficult to scale up for the large-scale energy system with more control options and operating conditions. Facing these challenges, this paper proposes a data-driven multi-agent power grid control scheme using a deep reinforcement learning (DRL) method. Specifically, the classic autonomous voltage control (AVC) problem is taken as an example and formulated as a Markov Game with a heuristic method to partition agents. Then, a multi-agent AVC (MA-AVC) algorithm based on a multi-agent deep deterministic policy gradient (MADDPG) method that features centralized training and decentralized execution is developed to solve the AVC problem. The proposed method can learn from scratch and gradually master the system operation rules by input and output data. In order to demonstrate the effectiveness of the proposed MA-AVC algorithm, comprehensive case studies are conducted on an Illinois 200-Bus system considering load/generation changes, N-1 contingencies, and weak centralized communication environment.


  • Open Access

    An Integrated Online Dynamic Security Assessment System for Improved Situational Awareness and Economic Operation

    by Haifeng Li, Ruisheng Diao, Xiaohu Zhang, Xi Lin, Xiao Lu, Di Shi, Zhiwei Wang, Lei Wang

    The ever-increasing penetration of centralized and distributed renewable energy, power electronics-based transmission equipment and loads, advanced protection and control systems, storage devices and new power market rules all contribute to the growing dynamics and stochastic behaviors being observed in today’s grid operation. Understanding operational risks and providing prompt control actions are of great importance to ensure secure and economic operation of a bulk power system. In this paper, a novel integrated online dynamic security assessment system (DSAS) is developed that intakes real-time EMS snapshots combining both bus/branch and node/breaker network models, performs dynamic contingency analysis under various conditions, calculates real-time transfer limits, and provides online control suggestions to mitigate operational risks. Several unique and innovative features are developed to address practical challenges, including: (1) real-time stitching of power flow information from both node/breaker and bus/branch models covering different geographical regions of interest; (2) online corrections and enhancements to power flow and dynamic models; (3) equipment-name-based modelling approaches for complex contingencies and control systems; and (4) distributed computing capabilities for significant computational speed enhancement. The developed DSAS has been deployed in the control center of State Grid Jiangsu Electric Power Company with hundreds of HVAC and 8 (U)HVDC transmission lines, which has been running reliably since Sept. 2018, achieving satisfactory performance in improving situational awareness and economic operation of the power grid.


  • Top Cited Papers

    Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations

    by Jiajun Duan, Di Shi, Ruisheng Diao, Haifeng Li, Zhiwei Wang, Bei Zhang, Desong Bian, Zhehan Yi

    In this letter, a novel autonomous control framework “Grid Mind” is proposed for the secure operation of power grids based on cutting-edge artificial intelligence (AI) technologies. The proposed platform provides a data-driven, model-free and closed-loop control agent trained using deep reinforcement learning (DRL) algorithms by interacting with massive simulations and/or real environment of a power grid. The proposed agent learns from scratch to master the power grid voltage control problem purely from data. It can make autonomous voltage control (AVC) strategies to support grid operators in making effective and timely control actions, according to the current system conditions detected by real-time measurements from supervisory control and data acquisition (SCADA) or phasor measurement units (PMUs). Two state-of-the-art DRL algorithms, namely deep Q-network (DQN) and deep deterministic policy gradient (DDPG), are proposed to formulate the AVC problem with performance compared. Case studies on a realistic 200-bus test system demonstrate the effectiveness and promising performance of the proposed framework.


  • Open Access

    Mathematical representation of the WECC composite load model

    by Zixiao Ma, Zhaoyu Wang, Yishen Wang, Ruisheng Diao, Di Shi

    The Western Electricity Coordinating Council (WECC) composite load model is a newly developed load model that has drawn great interest from the industry. To analyze its dynamic characteristics with both mathematical and engineering rigor, a detailed mathematical model is needed. Although WECC composite load model is available in commercial software as a module and its detailed block diagrams can be found in several public reports, there is no complete mathematical representation of the full model in literature. This paper addresses a challenging problem of deriving detailed mathematical representation of WECC composite load model from its block diagrams. In particular, for the first time, we have derived the mathematical representation of the new DER_A model. The developed mathematical model is verified using both Matlab and PSS/E to show its effectiveness in representing WECC composite load model. The derived mathematical representation serves as an important foundation for parameter identification, order reduction and other dynamic analysis.

  • Robust Time-Varying Synthesis Load Modeling in Distribution Networks Considering Voltage Disturbances

    by Mingjian Cui, Jianhui Wang, Yishen Wang, Ruisheng Diao, Di Shi

    Uncertain power sources are increasingly integrated into distribution networks and causing more challenges for the traditional load modeling. A variety of distributed load components present dynamic characteristics with time-varying parameters. Toward this end, this paper proposes a robust time-varying parameter identification (TVPI) method for synthesis load modeling (SLM) in distribution networks, including time-varying ZIP, induction motor, and equivalent impedance models. The nonlinear optimization model is developed and solved by the nonlinear least square (NLS) to find the minimum error between estimated outputs and measurements. To cope with TVPI deteriorated by voltage anomalies, dynamic programming is first used to detect anomalies. Then, a robust TVPI engine is designed to constrain the estimated time-varying parameters within a stable range. Furthermore, advanced tolerance thresholds are also required during iterations of NLS. Numerical simulations on the 9- and 129-bus distribution systems verify the effectiveness and robustness of the proposed TVPI method. Also, this method can be robust to the ambient noise of measurements.


  • Open Access

    Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning

    by Ruisheng Diao, Zhiwei Wang, Di Shi, Qianyun Chang, Jiajun Duan, Xiaohu Zhang

    Modern power grids are experiencing grand challenges caused by the stochastic and dynamic nature of growing renewable energy and demand response. Traditional theoretical assumptions and operational rules may be violated, which are difficult to be adapted by existing control systems due to the lack of computational power and accurate grid models for use in real time, leading to growing concerns in the secure and economic operation of the power grid. Existing operational control actions are typically determined offline, which are less optimized. This paper presents a novel paradigm, Grid Mind, for autonomous grid operational controls using deep reinforcement learning. The proposed AI agent for voltage control can learn its control policy through interactions with massive offline simulations, and adapts its behavior to new changes including not only load/generation variations but also topological changes. A properly trained agent is tested on the IEEE 14-bus system with tens of thousands of scenarios, and promising performance is demonstrated in applying autonomous voltage controls for secure grid operation.