Component-distinguishable Co-location and Resource Reclamation for High-throughput Computing
Open Access
- 13 February 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer Systems
- Vol. 42 (1-2) , 1-37
Abstract
Cloud service providers improve resource utilization by co-locating latency-critical (LC) workloads with best-effort batch (BE) jobs in datacenters. However, they usually treat multi-component LCs as monolithic applications and treat BEs as “second-class citizens” when allocating resources to them. Neglecting the inconsistent interference tolerance abilities of LC components and the inconsistent preemption loss of BE workloads can result in missed co-location opportunities for higher throughput. We present Rhythm , a co-location controller that deploys workloads and reclaims resources rhythmically for maximizing the system throughput while guaranteeing LC service’s tail latency requirement. The key idea is to differentiate the BE throughput launched with each LC component, that is, components with higher interference tolerance can be deployed together with more BE jobs. It also assigns different reclamation priority values to BEs by evaluating their preemption losses into a multi-level reclamation queue. We implement and evaluate Rhythm using workloads in the form of containerized processes and microservices. Experimental results show that it can improve the system throughput by 47.3%, CPU utilization by 38.6%, and memory bandwidth utilization by 45.4% while guaranteeing the tail latency requirement.Keywords
Funding Information
- National Key Research and Development Program of China (2022YFB4500702)
- Shandong Provincial Natural Science Foundation (ZR2022LZH018)
- National Natural Science Foundation of China (62141218, 62372322)
- Zhejiang Lab (2021DA0AM01/003)
This publication has 44 references indexed in Scilit:
- Predicting Cross-Core Performance Interference on Multicore Processors with Regression AnalysisIEEE Transactions on Parallel and Distributed Systems, 2015
- Bubble-fluxACM SIGARCH Computer Architecture News, 2013
- The tail at scaleCommunications of the ACM, 2013
- Law of the Minimum ParadoxesBulletin of Mathematical Biology, 2010
- Runtime measurements in the cloudProceedings of the VLDB Endowment, 2010
- Contention awareness and fault-tolerant scheduling for precedence constrained tasks in heterogeneous systemsParallel Computing, 2008
- StardustACM SIGMETRICS Performance Evaluation Review, 2006
- Discovering and exploiting program phasesIEEE Micro, 2003
- A survey of rollback-recovery protocols in message-passing systemsACM Computing Surveys, 2002
- TPC-W: a benchmark for e-commerceIEEE Internet Computing, 2002