Machine Learning Inference on Serverless Platforms Using Model Decomposition
Open Access
- 4 December 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Abstract
Serverless offers a scalable and cost-effective service model for users to run applications without focusing on underlying infrastructure or physical servers. While the Serverless architecture is not designed to address the unique challenges posed by resource-intensive workloads, e.g., Machine Learning (ML) tasks, it is highly scalable. Due to the limitations of Serverless function deployment and resource provisioning, the combination of ML and Serverless is a complex undertaking. We tackle this problem through decomposition of large ML models into smaller sub-models, referred to as slices. We set up ML inference tasks using these slices as a Serverless workflow, i.e., sequence of functions. Our experimental evaluations are performed on the Serverless offering by AWS for demonstration purposes, considering an open-source format for ML model representation, Open Neural Network Exchange. Achieved results portray that our decomposition method enables the execution of ML inference tasks on Serverless, regardless of the model size, benefiting from the high scalability of this architecture while lowering the strain on computing resources, such as required run-time memory.Keywords
Funding Information
- Horizon 2020 Framework Programme (860627)
This publication has 10 references indexed in Scilit:
- Serving distributed inference deep learning models in serverless computing2022
- Serverless Computing: A Survey of Opportunities, Challenges, and ApplicationsACM Computing Surveys, 2022
- λDNN: Achieving Predictable Distributed DNN Training With Serverless ArchitecturesIEEE Transactions on Computers, 2021
- A Survey of the Usages of Deep Learning for Natural Language ProcessingIEEE Transactions on Neural Networks and Learning Systems, 2020
- Toward Collaborative Inferencing of Deep Neural Networks on Internet-of-Things DevicesIEEE Internet of Things Journal, 2020
- DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge ClustersIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018
- Exploring Serverless Computing for Neural Network Training2018
- Serving Deep Learning Models in a Serverless Platform2018
- A review of deep learning in image recognition2017
- MoDNN: Local distributed mobile computing system for Deep Neural Network2017