Nightcore: efficient and scalable serverless computing for latency-sensitive, interactive microservices
Top Cited Papers
Open Access
- 17 April 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 152-166
Abstract
The microservice architecture is a popular software engineering approach for building flexible, large-scale online services. Serverless functions, or function as a service (FaaS), provide a simple programming model of stateless functions which are a natural substrate for implementing the stateless RPC handlers of microservices, as an alternative to containerized RPC servers. However, current serverless platforms have millisecond-scale runtime overheads, making them unable to meet the strict sub-millisecond latency targets required by existing interactive microservices. We present Nightcore, a serverless function runtime with microsecond-scale overheads that provides container-based isolation between functions. Nightcore’s design carefully considers various factors having microsecond-scale overheads, including scheduling of function requests, communication primitives, threading models for I/O, and concurrent function executions. Nightcore currently supports serverless functions written in C/C++, Go, Node.js, and Python. Our evaluation shows that when running latency-sensitive interactive microservices, Nightcore achieves 1.36×–2.93× higher throughput and up to 69% reduction in tail latency.Keywords
Funding Information
- NSF (National Science Foundation) (CNS-2008321)
- NSF (CNS-1900457)
This publication has 24 references indexed in Scilit:
- ZygOS2017
- Strata2017
- Occupy the cloud2017
- Attack of the killer microsecondsCommunications of the ACM, 2017
- CPI 22013
- The tail at scaleCommunications of the ACM, 2013
- Better never than late2011
- Capriccio2003
- SEDAACM SIGOPS Operating Systems Review, 2001
- Hints for computer system design1983