Empirically-Calibrated H100 Node Power Models for Reducing Uncertainty in AI Training Energy Estimation
Authors:
Preprint
Abstract
As AI's energy demand continues to grow, it is critical to enhance the understanding of characteristics of this demand, to improve grid infrastructure planning and environmental assessment. By combining empirical measurements from Brookhaven National Laboratory during AI training on 8-GPU H100 systems with open-source benchmarking data, we develop statistical models relating computational intensity to node-level power consumption. We measure the gap between manufacturer-rated thermal design power (TDP) and actual power demand during AI training. Our analysis reveals that even computationally intensive workloads operate at only 76% of the 10.2 kW TDP rating. Our architecture-specific model, calibrated to floating-point operations, predicts energy consumption with 11.4% mean absolute percentage error, significantly outperforming TDP-based approaches (27-37% error). We identified distinct power signatures between transformer and CNN architectures, with transformers showing characteristic fluctuations that may impact grid stability.Keywords
All Related Versions
This publication has 70 references indexed in Scilit:
- To better understand AI’s growing energy use, analysts need a data revolutionJoule, 2024
- Estimating the environmental impact of Generative-AI services using an LCA-based methodologyProcedia CIRP, 2024
- The growing energy footprint of artificial intelligenceJoule, 2023
- A systematic review of Green AIWIREs Data Mining and Knowledge Discovery, 2023
- Energy-aware systems for real-time job scheduling in cloud data centers: A deep reinforcement learning approachComputers and Electrical Engineering, 2022
- Cooling technologies for data centres and telecommunication base stations – A comprehensive reviewJournal of Cleaner Production, 2022
- Precise Energy Consumption Measurements of Heterogeneous Artificial Intelligence Workloads2022
- Assessing ICT global emissions footprint: Trends to 2040 & recommendationsJournal of Cleaner Production, 2018
- Optimizing performance-per-watt on GPUs in high performance computingComputer Science - Research and Development, 2015
- Electrical Grid and Supercomputing Centers: An Investigative Analysis of Emerging Opportunities and ChallengesInformatik Spektrum, 2014