Using reinforcement learning for optimizing heat pump control in a building model in Modelica
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
With the increasing share of renewable energy sources in the electricity grid comes the need to exploit the available flexibility at the demand side. Demand response programs seek to exploit the flexibility of consumers by motivating end-users to shift demand based on grid signals. An important source of flexibility available at residential consumers are Thermostatically Controlled Loads (TCLs). Additionally, recent advances within the reinforcement learning area have made it possible to apply this technique to a large range of problems. Driven by these promising examples, a Batch Reinforcement Learning (BRL) algorithm is applied to a TCL. An important property of the complex optimization problem studied here, is its partial observability. The main contribution of this paper is the application of BRL to a detailed building and heating system model, implemented in Modelica. A detailed TCL model allows to perform an in-depth analysis of the effects of partial observability on the performance of the chosen control strategy. Ultimately, this paper illustrates that Modelica can be used to provide a detailed environment for a BRL algorithm. At the end, the learned control policy has been compared with two other control policies. The obtained policy outperforms both and is economically feasible after a limited amount of training days.Keywords
This publication has 13 references indexed in Scilit:
- Experimental analysis of data-driven control for a building heating systemSustainable Energy, Grids and Networks, 2016
- Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement LearningIEEE Transactions on Smart Grid, 2016
- Power system flexibility need induced by wind and solar power intermittency on time scales of 1–14 daysRenewable Energy, 2015
- Simulating occupancy in office buildings with non-homogeneous Markov chains for Demand Response analysis2015
- Reinforcement Learning of Heuristic EV Fleet Charging in a Day-Ahead Electricity MarketIEEE Transactions on Smart Grid, 2015
- Forecasting household consumer electricity load profiles with a combined physical and behavioral approachApplied Energy, 2014
- Use of model predictive control and weather forecasts for energy efficient building climate controlEnergy and Buildings, 2012
- Day-Ahead Electricity Price Forecasting Using the Wavelet Transform and ARIMA ModelsIEEE Transactions on Power Systems, 2005
- Neural Fitted Q Iteration – First Experiences with a Data Efficient Neural Reinforcement Learning Method2005
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998