Fine-grained accelerator partitioning for Machine Learning and Scientific Computing in Function as a Service Platform
Open Access
- 12 November 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1606-1613
Abstract
Function-as-a-service (FaaS) is a promising execution environment for high-performance computing (HPC) and machine learning (ML) applications as it offers developers a simple way to write and deploy programs. Nowadays, GPUs and other accelerators are indispensable for HPC and ML workloads. These accelerators are expensive to acquire and operate; consequently, multiplexing them can increase their financial profitability. However, we have observed that state-of-the-art FaaS frameworks usually treat accelerator as a single device to run single workload and have little support for multiplexing accelerators. In this work, we have presented techniques to multiplex GPUs with Parsl, a popular FaaS framework. We demonstrate why GPU multiplexing is beneficial for certain applications and how we have implemented GPU multiplexing in Parsl. With our enhancements, we show up to 60% lower task completion time and 250% improvement in the inference throughput of a large language model when multiplexing a GPU compared to running a single instance without multiplexing. We plan to extend the support for GPU multiplexing in FaaS platforms by tackling the challenges of changing compute resources in the partition and approximating how to right-size a GPU partition for a function.Keywords
This publication has 20 references indexed in Scilit:
- Colmena: Scalable Machine-Learning-Based Steering of Ensemble Simulations for High Performance Computing2021
- ECML: Improving Efficiency of Machine Learning in Edge Clouds2020
- Machine Learning at the Edge: Efficient Utilization of Limited CPU/GPU Resources by Multiplexing2020
- GSLICE2020
- funcX: A Federated Function Serving Fabric for Science2020
- XSP: Across-Stack Profiling and Analysis of Machine Learning Models on GPUs2020
- TrIMS: Transparent and Isolated Model Sharing for Low Latency Deep Learning Inference in Function-as-a-Service2019
- Parsl2019
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- ImageNet: A large-scale hierarchical image database2009