Marco Biemann
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Marco Biemann has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Fabian Scheller
Institute for Infrastructure and Resources Management (IIRM), Leipzig University, Germany
2 shared publications - Lizhen Huang
Norwegian University of Science and Technology (NTNU)
2 shared publications - Xiufeng Liu
Technical University of Denmark, Lyngby, Denmark
2 shared publications - Henrik Madsen
Technical University of Denmark
1 shared publication Philipp Andreas Gunkel
Department of Technology, Management and Economics, Section of Energy Economics and Modelling, Technical University of Denmark, Lyngby, Denmark
1 shared publication- Uffe Høgsbro Thygesen
DTU Compute Technical University of Denmark DK-2800 Kongens Lyngby Denmark
1 shared publication
Latest Publications
- Open Access
- 22 July 2026
- Published by Elsevier in Energy Conversion and Management
- Vol. 367
Grey-box model of a short-term-operated pit thermal energy storage system
- 22 July 2026
- Published by Elsevier in Energy Conversion and Management
- Vol. 367
Open AccessGrey-box model of a short-term-operated pit thermal energy storage system
- 30 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 10 (15) , 13876-13894
Data Center HVAC Control Harnessing Flexibility Potential via Real-Time Pricing Cost Optimization Using Reinforcement Learning
With increasing electricity prices, cost savings through load shifting are becoming increasingly important for energy end users. While dynamic pricing encourages customers to shift demand to low price periods, the non-stationary and highly volatile nature of electricity prices poses a significant challenge to energy management systems. In this paper, we investigate the flexibility potential of data centres by optimising heating, ventilation, and air conditioning systems with a general model-free reinforcement learning approach. Since the soft actor-critic algorithm with feed-forward networks did not work satisfactorily in this scenario, we propose instead a parameterisation with a recurrent neural network architecture to successfully handle spot-market price data. The past is encoded into a hidden state, which provides a way to learn the temporal dependencies in the observations and highly volatile rewards. The proposed method is then evaluated in experiments on a simulated data centre. Considering real temperature and price signals over multiple years, the results show a cost reduction compared to a proportional, integral and derivative controller while maintaining the temperature of the data centre within the desired operating ranges. In this context, this work demonstrates an innovative and applicable reinforcement learning approach that incorporates complex economic objectives into agent decision-making. The proposed control method can be integrated into various Internet of things-based smart building solutions for energy management.
- 30 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 10 (15) , 13876-13894
Data Center HVAC Control Harnessing Flexibility Potential via Real-Time Pricing Cost Optimization Using Reinforcement Learning
With increasing electricity prices, cost savings through load shifting are becoming increasingly important for energy end users. While dynamic pricing encourages customers to shift demand to low price periods, the non-stationary and highly volatile nature of electricity prices poses a significant challenge to energy management systems. In this paper, we investigate the flexibility potential of data centres by optimising heating, ventilation, and air conditioning systems with a general model-free reinforcement learning approach. Since the soft actor-critic algorithm with feed-forward networks did not work satisfactorily in this scenario, we propose instead a parameterisation with a recurrent neural network architecture to successfully handle spot-market price data. The past is encoded into a hidden state, which provides a way to learn the temporal dependencies in the observations and highly volatile rewards. The proposed method is then evaluated in experiments on a simulated data centre. Considering real temperature and price signals over multiple years, the results show a cost reduction compared to a proportional, integral and derivative controller while maintaining the temperature of the data centre within the desired operating ranges. In this context, this work demonstrates an innovative and applicable reinforcement learning approach that incorporates complex economic objectives into agent decision-making. The proposed control method can be integrated into various Internet of things-based smart building solutions for energy management.
- Top Cited PapersOpen Access
- 1 September 2021
- Published by Elsevier in Applied Energy
- Vol. 298
Experimental evaluation of model-free reinforcement learning algorithms for continuous HVAC control
- 1 September 2021
- Published by Elsevier in Applied Energy
- Vol. 298
Top Cited PapersOpen AccessExperimental evaluation of model-free reinforcement learning algorithms for continuous HVAC control