Design and Operation of Shared Machine Learning Clusters on Campus
Open Access
- 30 March 2025
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 295-310
Abstract
The rapid advancement of large machine learning (ML) models has driven universities worldwide to invest heavily in GPU clusters. Effectively sharing these resources among multiple users is essential for maximizing both utilization and accessibility. However, managing shared GPU clusters presents significant challenges, ranging from system configuration to fair resource allocation among users. This paper introduces SING, a full-stack solution tailored to simplify shared GPU cluster management. Aimed at addressing the pressing need for efficient resource sharing with limited staffing, SING enhances operational efficiency by reducing maintenance costs and optimizing resource utilization. We provide a comprehensive overview of its four extensible architectural layers, explore the features of each layer, and share insights from real-world deployment, including usage patterns and incident management strategies. As part of our commitment to advancing shared ML cluster management, we open-source SING's resources to support the development and operation of similar systems.Keywords
Funding Information
- Research Grants Council, University Grants Committee (Hong Kong RGC TRS T41-603/20-R)
This publication has 8 references indexed in Scilit:
- Sia: Heterogeneity-aware, goodput-optimized ML-cluster scheduling2023
- Lucid: A Non-intrusive, Scalable and Interpretable Scheduler for Deep Learning Training Jobs2023
- LiteFlow2022
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Real-World, Self-Hosted Kubernetes Experience2021
- CoDeNet: Efficient Deployment of Input-Adaptive Object Detection on Embedded FPGAs2021
- Data center TCP (DCTCP)2010
- Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfillingIEEE Transactions on Parallel and Distributed Systems, 2001