Safe Exploration Reinforcement Learning for Load Restoration using Invalid Action Masking
Authors:
- 16 July 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Abstract
This paper addresses the load restoration problem after a power outage event. Our primary proposed methodology uses a multi-agent reinforcement learning method to make the optimal sequential decisions on picking up critical loads. Typically, a negative reward is provided to discourage the agents from selecting decisions that violate physical constraints during the restoration process. However, the main disadvantage of this approach is its difficulty in applying it to large-scale systems due to the curse of dimensionality. This paper introduces the invalid action masking technique to overcome this limitation. The features of this technique include zero physical constraint violations, reduced training time, and stabilization of the exploration process. Simulation results are performed in IEEE 13-node and IEEE 123-node systems showing the performance of the proposed algorithm in comparison to the conventional approach both in terms of restored power and learning curve.Keywords
Funding Information
- Office of Naval Research
This publication has 15 references indexed in Scilit:
- Constraint-Guided Reinforcement Learning: Augmenting the Agent-Environment-Interaction2021
- Distribution System Resilience Under Asynchronous Information Using Deep Reinforcement LearningIEEE Transactions on Power Systems, 2021
- Resilient Load Restoration in Microgrids Considering Mobile Energy Storage Fleets: A Deep Reinforcement Learning Approach2020
- Grandmaster level in StarCraft II using multi-agent reinforcement learningNature, 2019
- A Reinforcement Learning Approach to Solve Service Restoration and Load Management Simultaneously for Distribution NetworksIEEE Access, 2019
- New multi‐stage restoration method for distribution networks with DGsIET Generation, Transmission & Distribution, 2018
- Optimization of uncertainty load restoration strategy based on assessment of Conditional Value-at-Risk2016
- Human-level control through deep reinforcement learningNature, 2015
- Dynamic reconfiguration of shipboard power systems using reinforcement learningIEEE Transactions on Power Systems, 2012
- Shipboard power system reconfiguration using reinforcement learning2010