SiloD: A Co-design of Caching and Scheduling for Deep Learning Clusters
Open Access
- 8 May 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 883-898
Abstract
Deep learning training on cloud platforms usually follows the tradition of the separation of storage and computing. The training executes on a compute cluster equipped with GPUs/TPUs while reading data from a separate cluster hosting the storage service. To alleviate the potential bottleneck, a training cluster usually leverages its local storage as a cache to reduce the remote IO from the storage cluster. However, existing deep learning schedulers do not manage storage resources thus fail to consider the diverse caching effects across different training jobs. This could degrade scheduling quality significantly. To address this issue, we present SiloD, a scheduling framework that co-designs the cluster scheduler and the cache subsystems for deep learning training. SiloD treats cache and remote IO as first-class resources and can integrate different state-of-the-art deep learning scheduling policies in a unified scheduling framework. To achieve this, SiloD develops an enhanced job performance estimator to help different schedulers to jointly consider the impact of storage and compute resource allocation while preserving their respective scheduling objectives. The SiloD-enhanced performance estimator leverages the unique data access pattern of deep learning training to develop a closed-form analytic model that captures the diverse cache / remote IO requirements from different training jobs. Evaluations show that SiloD improves the average job completion time, cluster utilization, and fairness by up to 7.4x, 2.57x, and 1.89x, respectively, compared to different combinations of cache systems and cluster schedulers where they operate independently.Keywords
Funding Information
- National Key Research and Development Program of China (2021ZD0110202)
This publication has 26 references indexed in Scilit:
- Rethinking the Inception Architecture for Computer Vision2016
- Borg, Omega, and KubernetesCommunications of the ACM, 2016
- TetriSched2016
- Multi-resource packing for cluster schedulersACM SIGCOMM Computer Communication Review, 2014
- Small cache, big effect2011
- Delay scheduling2010
- Role of Aging, Frequency, and Size in Web Cache Replacement Policies2001
- Evaluating content management techniques for Web proxy cachesACM SIGMETRICS Performance Evaluation Review, 2000
- Caching on the World Wide WebIEEE Transactions on Knowledge and Data Engineering, 1999
- The LRU-K page replacement algorithm for database disk bufferingACM SIGMOD Record, 1993