Phyllis: Physics-Informed Lifelong Reinforcement Learning for Data Center Cooling Control
Open Access
- 16 June 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 114-126
Abstract
Deep reinforcement learning (DRL) has shown good performance in data center cooling control for improving energy efficiency. The main challenge in deploying the DRL agent to real-world data centers is how to quickly adapt the agent to the ever-changing system with thermal safety compliance. Existing approaches rely on DRL’s native fine-tuning or a learned data-driven dynamics model to assist the adaptation. However, they require long-term unsafe exploration before the agent or the model can capture a new environment. This paper proposes Phyllis, a physics-informed reinforcement learning approach to assist the DRL agent’s lifelong learning under evolving data center environment. Phyllis first identifies a transition model to capture the data hall thermodynamics in the offline stage. When the environment changes in the online stage, Phyllis assists the adaptation by i) supervising safe data collection with the identified transition model, ii) fitting power usage and residual thermal models, iii) pretraining the agent by interacting with these models, and iv) deploying the agent for further fine-tuning. Phyllis uses known physical laws to inform the transition and power models for improving the extrapolation ability to unseen states. Extensive evaluation for two simulated data centers with different system changes shows that Phyllis saves 5.7% to 13.8% energy usage compared with feedback cooling control and adapts to new environments 8x to 10x faster than fine-tuning with at most 0.74°C temperature overshoot.Keywords
Funding Information
- National Natural Science Foundation, China (62262026 and GJJ211111)
- Nanyang Technological University, Singapore (NTU?ACE2020-01)
- Ministry of Education, Singapore (RT14/22 and RG96/20)
- National Research Foundation, Singapore (NRF2020NRF-CG001-027)
This publication has 22 references indexed in Scilit:
- One for Many2020
- Kalibre2020
- A Relearning Approach to Reinforcement Learning for control of Smart BuildingsAnnual Conference of the PHM Society, 2020
- Control of Air Free-Cooled Data Centers in Tropics via Deep Reinforcement Learning2019
- Building HVAC Scheduling Using Reinforcement Learning via Neural Network Based Model Approximation2019
- Transforming Cooling Optimization for Green Data Center via Deep Reinforcement LearningIEEE Transactions on Cybernetics, 2019
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equationsJournal of Computational Physics, 2019
- Data Driven Chiller Plant Energy Optimization with Domain Knowledge2017
- On Global Electricity Usage of Communication Technology: Trends to 2030Challenges, 2015
- PID autotuner and its application in HVAC systems2001