Power Consumption Optimization of GPU Server With Offline Reinforcement Learning
Open Access
- 14 May 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 85826-85837
Abstract
The rapid growth of artificial intelligence (AI) has led to increased reliance on power-intensive Graphics Processing Units (GPUs), which are essential for training and deploying large-scale models. However, the escalating energy demands of AI workloads pose sustainability challenges, necessitating efficient power management strategies to reduce carbon footprints. Optimizing GPU server power consumption is complex due to the interdependence of various components. Conventional methods often involve trade-offs: increasing fan speed enhances cooling but raises overall power usage, whereas lowering GPU clock frequencies conserves energy at the cost of longer computation times. To address these challenges, we propose a data-driven optimization framework based on offline reinforcement learning (RL). Our approach collects operational data from a custom-designed workload that simulates varying server loads, capturing key metrics such as power consumption, temperature, and core frequency. The reward function balances power efficiency with performance. The reinforcement learning agent learns from pre-collected server logs, enabling intelligent real-time GPU clock control decisions without costly live experiments. Additionally, periodic fan speed adjustments and pre-training of the Q-network further enhance overall efficiency. Experimental results demonstrate that our method reduces power consumption by 3.62% while improving computation time by 1.51% for synthetic workloads. For LLaMA-2 fine-tuning, power consumption decreases by 6.40% with only a minor 1.27% increase in computation time, demonstrating its practical effectiveness. Our framework was validated on the latest NVIDIA L40S GPU, demonstrating its compatibility with cutting-edge hardware.Keywords
Funding Information
- KT Corporation (KT) Korea Advanced Institute of Science and Technology (KAIST) Open Research and Development Project funded by KT
This publication has 16 references indexed in Scilit:
- A multi-agent reinforcement learning-based method for server energy efficiency optimization combining DVFS and dynamic fan controlSustainable Computing: Informatics and Systems, 2024
- Going green: optimizing GPUs for energy efficiency through model-steered auto-tuning2022
- Energy-Efficient Computation Offloading With DVFS Using Deep Reinforcement Learning for Time-Critical IoT Applications in Edge ComputingIEEE Internet of Things Journal, 2022
- Multi-Core Power Management through Deep Reinforcement Learning2021
- GPU Energy Consumption Optimization With a Global-Based Neural Network MethodIEEE Access, 2019
- Optimizing performance-per-watt on GPUs in high performance computingComputer Science - Research and Development, 2015
- GreenGPU: A Holistic Approach to Energy Efficiency in GPU-CPU Heterogeneous Architectures2012
- An Instruction-Level Energy Estimation and Optimization Methodology for GPU2011
- SPRAT: Runtime processor selection for energy-aware computing2008
- Computer Architecture Techniques for Power-Efficiency2008