HPC Digital Twins for Evaluating Scheduling Policies, Incentive Structures and their Impact on Power and Cooling
Authors:
- Matthias MaiterthMMMatthias Maiterth
- Wesley H. BrewerWBWesley H. Brewer
- Jaya S. KuruvellaJKJaya S. Kuruvella
- Arunavo DeyADArunavo Dey
- Tanzima Z. IslamTITanzima Z. Islam
- Kevin MenearKMKevin Menear
- Dmitry DuplyakinDDDmitry Duplyakin
- Rashadul KabirRKRashadul Kabir
- Tapasya PatkiTPTapasya Patki
- Terry JonesTJTerry Jones
- Feiyi WangFWFeiyi Wang
Preprint
Abstract
Schedulers are critical for optimal resource utilization in high-performance computing. Traditional methods to evaluate schedulers are limited to post-deployment analysis, or simulators, which do not model associated infrastructure. In this work, we present the first-of-its-kind integration of scheduling and digital twins in HPC. This enables what-if studies to understand the impact of parameter configurations and scheduling decisions on the physical assets, even before deployment, or regarching changes not easily realizable in production. We (1) provide the first digital twin framework extended with scheduling capabilities, (2) integrate various top-tier HPC systems given their publicly available datasets, (3) implement extensions to integrate external scheduling simulators. Finally, we show how to (4) implement and evaluate incentive structures, as-well-as (5) evaluate machine learning based scheduling, in such novel digital-twin based meta-framework to prototype scheduling. Our work enables what-if scenarios of HPC systems to evaluate sustainability, and the impact on the simulated system.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- F-DATA: A Fugaku Workload Dataset for Job-centric Predictive Modelling in HPC SystemsScientific Data, 2025
- An optimization-centric review on integrating artificial intelligence and digital twin technologies in manufacturingEngineering Optimization, 2025
- A Fast Simulator to Enable HPC Scheduling Strategy Comparisons2023
- Experience and Practice of Batch Scheduling on Leadership Supercomputers at Argonne2018
- Digital Twin Data Modeling with AutomationML and a Communication Methodology for Data ExchangeIFAC-PapersOnLine, 2016
- Versatile, scalable, and accurate simulation of distributed applications and platformsJournal of Parallel and Distributed Computing, 2014
- Simbatch: An API for Simulating and Predicting the Performance of Parallel Resources Managed by Batch Systems2009
- GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computingConcurrency and Computation: Practice and Experience, 2002
- Scheduling with batching: A reviewEuropean Journal of Operational Research, 2000
- The EASY — LoadLeveler API project1996