Distributed Deep Reinforcement Learning for Intelligent Load Scheduling in Residential Smart Grids
- 6 July 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 17 (4) , 2752-2763
Abstract
The power consumption of households has been constantly growing over the years. To cope with this growth, intelligent management of the consumption profile of the households is necessary, such that the households can save the electricity bills, and the stress to the power grid during peak hours can be reduced. However, implementing such a method is challenging due to the existence of randomness in the electricity price and the consumption of the appliances. To address this challenge, we employ a model-free method for the households which works with limited information about the uncertain factors. More specifically, the interactions between households and the power grid can be modeled as a non-cooperative stochastic game, where the electricity price is viewed as a stochastic variable. To search for the Nash equilibrium (NE) of the game, we adopt a method based on distributed deep reinforcement learning. Also, the proposed method can preserve the privacy of the households. We then utilize real-world data from Pecan Street Inc., which contains the power consumption profile of more than 1,000 households, to evaluate the performance of the proposed method. In average, the results reveal that we can achieve around 12% reduction on peak-to-average ratio (PAR) and 11% reduction on load variance. With this approach, the operation cost of the power grid and the electricity cost of the households can be reduced.Keywords
All Related Versions
Funding Information
- Norges Forskningsråd (275106, 287412, 267967)
This publication has 30 references indexed in Scilit:
- Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement LearningIEEE Transactions on Smart Grid, 2016
- Dynamic Pricing and Energy Consumption Scheduling With Reinforcement LearningIEEE Transactions on Smart Grid, 2015
- Optimal Industrial Load Control in Smart GridIEEE Transactions on Smart Grid, 2015
- Demand Response Management in the Smart Grid in a Large Population RegimeIEEE Transactions on Smart Grid, 2015
- Human-level control through deep reinforcement learningNature, 2015
- Optimal Demand Response Using Device-Based Reinforcement LearningIEEE Transactions on Smart Grid, 2015
- A Distributed Algorithm for Managing Residential Demand Response in Smart GridsIEEE Transactions on Industrial Informatics, 2014
- Dependable Demand Response Management in the Smart Grid: A Stackelberg Game ApproachIEEE Transactions on Smart Grid, 2013
- Reinforcement Learning Versus Model Predictive Control: A Comparison on a Power System ProblemIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008
- The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement LearningSIAM Journal on Control and Optimization, 2000