Deep Reinforcement Learning for room temperature control: a black-box pipeline from data to policies
Authors:
Open Access
- 1 November 2021
- conference paper
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 2042 (1)
Abstract
Deep Reinforcement Learning (DRL) recently emerged as a possibility to control complex systems without the need to model them. However, since weeks long experiments are needed to assess the performance of a building controller, people still have to rely on accurate simulation environments to train and tune DRL agents in tractable amounts of time before deploying them, shifting the burden back to the original issue of designing complex models. In this work, we show that it is possible to learn control policies on simple black-box linear room temperature models, thereby alleviating the heavy engineering usually required to build accurate surrogates. We develop a black-box pipeline, where historical data is taken as input to produce room temperature control policies. The trained DRL agents are capable of beating industrial rule-based controllers both in terms of energy consumption and comfort satisfaction, using novel penalties to introduce expert knowledge, i.e. to incentivize agents to follow expected behaviors, in the reward function. Moreover, one of the best agents was deployed on a real building for one week and was able to save energy while maintaining adequate comfort levels, indicating that low-complexity models might be enough to learn control policies that perform well on real buildings.Keywords
This publication has 8 references indexed in Scilit:
- Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: Simulations and experimentsApplied Energy, 2022
- Reinforcement learning for building controls: The opportunities and challengesApplied Energy, 2020
- Real-Time Residential Demand ResponseIEEE Transactions on Smart Grid, 2020
- Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learningEnergy and Buildings, 2019
- Energy optimization associated with thermal comfort and indoor air control via a deep reinforcement learning algorithmBuilding and Environment, 2019
- Reinforcement learning for demand response: A review of algorithms and modeling techniquesApplied Energy, 2019
- Data-driven model predictive control using random forests for building energy optimization and climate controlApplied Energy, 2018
- Model Predictive Control (MPC) for Enhancing Building and HVAC System Energy Efficiency: Problem Formulation, Applications and OpportunitiesEnergies, 2018