Kernel-as-a-Service
Authors:
- Tobias PfandzelterTPTobias Pfandzelter
- Aditya DhakalADAditya Dhakal
- Eitan FrachtenbergEFEitan Frachtenberg
- Sai Rahul ChalamalasettiSCSai Rahul Chalamalasetti
- Darel EmmotDEDarel Emmot
- Ninad HogadeNHNinad Hogade
- Rolando Pablo Hong EnriquezRERolando Pablo Hong Enriquez
- Gourav RattihalliGRGourav Rattihalli
- David BermbachDBDavid Bermbach
- Dejan MilojicicDMDejan Milojicic
Open Access
- 27 November 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 192-206
Abstract
With the slowing of Moore's law and decline of Dennard scaling, computing systems increasingly rely on specialized hardware accelerators in addition to general-purpose compute units. Increased hardware heterogeneity necessitates disaggregating applications into workflows of fine-grained tasks that run on a diverse set of CPUs and accelerators. Current accelerator delivery models cannot support such applications efficiently, as (1) the overhead of managing accelerators erases performance benefits for fine-grained tasks; (2) exclusive accelerator use per task leads to underutilization; and (3) specialization increases complexity for developers. We propose adopting concepts from Function-as-a-Service (FaaS), which has solved these challenges for general-purpose CPUs in cloud computing. Kernel-as-a-Service (KaaS) is a novel serverless programming model for generic compute accelerators that aids heterogeneous workflows by combining the ease-of-use of higher-level abstractions with the performance of low-level hand-tuned code. We evaluate KaaS with a focus on the breadth of the idea and its generality to diverse architectures rather than on an in-depth implementation for a single accelerator. Using proof-of-concept prototypes, we show that this programming model provides performance, performance efficiency, and ease-of-use benefits across a diverse range of compute accelerators. Despite increased levels of abstraction, when compared to a naive accelerator implementation, KaaS reduces completion times for fine-grained tasks by up to 96.0% (GPU), 68.4% (FPGA), 98.6% (TPU), and 34.9% (QPU) in our experiments.Keywords
This publication has 69 references indexed in Scilit:
- FLEPACM SIGPLAN Notices, 2017
- Impact of Shutdown Techniques for Energy-Efficient Cloud Data Centers2016
- Deep Residual Learning for Image Recognition2016
- Numba2015
- Simultaneous Multikernel: Fine-Grained Sharing of GPUsIEEE Computer Architecture Letters, 2015
- A variational eigenvalue solver on a photonic quantum processorNature Communications, 2014
- A review of Genetic Algorithms and Parallel Genetic Algorithms on Graphics Processing Unit (GPU)2013
- Medical Image AnalysisIEEE Pulse, 2011
- GPU Resource Sharing and Virtualization on High Performance Computing Systems2011
- Understanding the efficiency of GPU algorithms for matrix-matrix multiplication2004