Rhythm
Authors:
- Yungang BaoYBYungang Bao
Open Access
- 15 April 2020
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-17
Abstract
Cloud service providers improve resource utilization by co-locating latency-critical (LC) workloads with best-effort batch (BE) jobs in datacenters. However, they usually treat an LC workload as a whole when allocating resources to BE jobs and neglect the different features of components of an LC workload. This kind of coarse-grained co-location method leaves a significant room for improvement in resource utilization. Based on the observation of the inconsistent interference tolerance abilities of different LC components, we propose a new abstraction called Servpod, which is a collection of a LC parts that are deployed on the same physical machine together, and show its merits on building a fine-grained co-location framework. The key idea is to differentiate the BE throughput launched with each LC Servpod, i.e., Servpod with high interference tolerance ability can be deployed along with more BE jobs. Based on Servpods, we present Rhythm, a co-location controller that maximizes the resource utilization while guaranteeing LC service's tail latency requirement. It quantifies the interference tolerance ability of each servpod through the analysis of tail-latency contribution. We evaluate Rhythm using LC services in forms of containerized processes and microservices, and find that it can improve the system throughput by 31.7%, CPU utilization by 26.2%, and memory bandwidth utilization by 34% while guaranteeing the SLA (service level agreement).Keywords
Funding Information
- National Natural Science Foundation of China (61872265)
- the new Generation of Artificial Intelligence Sci- ence and Technology Major Project of Tianjin (18ZXZNGX00190, 19ZXZNGX00010)
- National Key Research and Development Program of China (2016YFB1000205)
This publication has 47 references indexed in Scilit:
- Predicting Cross-Core Performance Interference on Multicore Processors with Regression AnalysisIEEE Transactions on Parallel and Distributed Systems, 2015
- Supporting Differentiated Services in Computers via Programmable Architecture for Resourcing-on-Demand (PARD)ACM SIGARCH Computer Architecture News, 2015
- Ubik2014
- Bubble-fluxACM SIGARCH Computer Architecture News, 2013
- CPI 22013
- The tail at scaleCommunications of the ACM, 2013
- Law of the Minimum ParadoxesBulletin of Mathematical Biology, 2010
- Runtime measurements in the cloudProceedings of the VLDB Endowment, 2010
- StardustACM SIGMETRICS Performance Evaluation Review, 2006
- TPC-W: a benchmark for e-commerceIEEE Internet Computing, 2002