District Cooling System Control for Providing Operating Reserve Based on Safe Deep Reinforcement Learning
- 18 January 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Power Systems
- Vol. 39 (1) , 40-52
Abstract
Heating, ventilation, and air conditioning (HVAC) systems are well proved to be capable to provide operating reserve for power systems. As a type of large-capacity and energy-efficient HVAC system (up to 100 MW), district cooling system (DCS) is emerging in modern cities and has huge potential to be regulated as a flexible load. However, strategically controlling a DCS to provide flexibility is challenging, because one DCS services multiple buildings with complex thermal dynamics and uncertain cooling demands. Improper control may lead to significant thermal discomfort and even deteriorate the power system's operation security. To address the above issues, we propose a model-free control strategy based on the deep reinforcement learning (DRL) without the requirement of accurate system model and uncertainty distribution. To avoid damaging “trial & error” actions that may violate the system's operation security during the training process, we further propose a safe layer combined to the DDPG to guarantee the satisfaction of critical constraints, forming a safe-DDPG scheme. Moreover, after providing operating reserve, DCS increases power and tries to recover all the buildings' temperature back to set values, which may cause an instantaneous peak-power rebound and bring a secondary negative impact on power systems. Therefore, we design a self-adaption reward function within the proposed safe-DDPG scheme to constrain the peak-power effectively. Numerical studies based on a realistic DCS demonstrate the effectiveness of the proposed methods.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (52007200)
- Science and Technology Development Fund (SKL-IOTSC(UM)-2021-2023, 0003/2020/AKP))
This publication has 28 references indexed in Scilit:
- A model for the optimal design and management of a cogeneration system with energy storageEnergy and Buildings, 2016
- Ice Storage Air-Conditioning System Simulation with Dynamic Electricity Pricing: A Demand Response StudyEnergies, 2016
- Evaluating the Magnitude and Duration of Cold Load Pick-up on Residential Distribution FeedersUsing Multi-State Load ModelsIEEE Transactions on Power Systems, 2015
- Reconstructing Operating Reserve: Flexibility for Sustainable Power SystemsIEEE Transactions on Sustainable Energy, 2015
- Human-level control through deep reinforcement learningNature, 2015
- Theory and applications of HVAC control systems – A review of model predictive control (MPC)Building and Environment, 2014
- Demand response and smart grids—A surveyRenewable and Sustainable Energy Reviews, 2014
- Markov Decision Processes: A Tool for Sequential Decision Making under UncertaintyMedical Decision Making, 2009
- Evaluation of Reinforcement Learning for Optimal Control of Building Active and Passive Thermal Storage InventoryJournal of Solar Energy Engineering, 2006
- Heat exchange design handbookInternational Journal of Heat and Fluid Flow, 1983