A Physics-Informed Reinforcement Learning Framework for HVAC Optimization: Thermodynamically-Constrained Deep Deterministic Policy Gradients with Simulation-Based Validation
Open Access
Abstract
This paper presents a physics-informed reinforcement learning framework that embeds thermodynamic constraints directly into the policy network of a continuous control agent for HVAC optimization. We introduce a Thermodynamically-Constrained Deep Deterministic Policy Gradient (TC-DDPG) algorithm that operates on continuous actions and enforces physical feasibility through a differentiable constraint layer coupled with physics-regularized loss functions. In a simulation-based evaluation using a custom Python multi-zone resistance-capacitance (RC) thermal model, the proposed method achieves a 34.7% reduction in annual HVAC electricity consumption relative to a rule-based baseline (95% CI: 31.2–38.1%, n = 50 runs) and outperforms standard DDPG by 16.1 percentage points. Thermal comfort during occupied hours maintains PMV ∈ [−0.5, 0.5] for 98.3% of operational time, peak demand decreases by 35.8%, and simulated coefficient of performance (COP) improves from 2.87 ± 0.08 to 4.12 ± 0.10. Physics constraint violations are reduced by approximately 98.6% compared to unconstrained DDPG, demonstrating the effectiveness of architectural enforcement mechanisms within the simulation environment. We present a reference prototype and commit to a future public release of the code, configurations, and hyperparameters sufficient to reproduce the reported results. The paper explicitly addresses the limitations of simulation-based studies and presents a staged roadmap toward hardware-in-the-loop testing and pilot deployments in real buildings.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Physics-informed modularized neural network for advanced building control by deep reinforcement learningAdvances in Applied Energy, 2025
- An experimental evaluation of deep reinforcement learning algorithms for HVAC controlArtificial Intelligence Review, 2024
- Ten questions concerning reinforcement learning for building energy managementBuilding and Environment, 2023
- A reinforcement learning-enabled iterative learning control strategy of air-conditioning systems for building energy saving by shortening the morning start periodApplied Energy, 2023
- Reinforcement learning for building controls: The opportunities and challengesApplied Energy, 2020
- All you need to know about model predictive control for buildingsAnnual Reviews in Control, 2020
- A review of reinforcement learning for autonomous building energy managementComputers and Electrical Engineering, 2019
- Reinforcement learning for demand response: A review of algorithms and modeling techniquesApplied Energy, 2019
- Ten questions concerning model predictive control for energy efficient buildingsBuilding and Environment, 2016
- Review of modeling methods for HVAC systemsApplied Thermal Engineering, 2014