Derm: SLA-aware Resource Management for Highly Dynamic Microservices
Authors:
- 29 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 424-436
Abstract
Ensuring efficient resource allocation while providing service level agreement (SLA) guarantees for end-to-end (E2E) latency is crucial for microservice applications. Although existing studies have made significant contributions towards achieving this objective, they primarily concentrate on static graphs. However, microservice graphs are inherently dynamic during runtime in production environments, necessitating more effective and scalable resource management solutions.In this paper, we present Derm, a new resource management system designed for microservice applications with highly dynamic graphs. Our principal finding is that prioritizing different microservice graphs can lead to a substantial reduction in resource allocation. To take advantage of this opportunity, we develop three main components. The first is a performance model that describes uncertainties of microservice latency through a conditional exponential distribution. The second is a probabilistic quantification of the dynamics of microservice graphs. The third is an optimization method for adjusting the resource allocation of microservices to minimize resource usage. We evaluate Derm in our cluster using real microservice benchmarks and production traces. The results highlight that Derm reduces the resource usage by $68.4 \%$ and lowers SLA violation probability by $6.7 \times$, compared to existing approaches.
Keywords
Funding Information
- Science and Technology Development Fund
This publication has 30 references indexed in Scilit:
- An In-Depth Study of Microservice Call Graph and Runtime PerformanceIEEE Transactions on Parallel and Distributed Systems, 2022
- Q-Zilla: A Scheduling Framework and Core Microarchitecture for Tail-Tolerant Microservices2020
- ATOM: Model-Driven Autoscaling for Microservices2019
- MIRAS: Model-based Reinforcement Learning for Microservice Resource Allocation over Scientific Workflows2019
- Microscaler: Automatic Scaling for Microservices with an Online Learning Approach2019
- μ Suite: A Benchmark Suite for Microservices2018
- M/M/1 queue in two alternating environments and its heavy traffic approximationJournal of Mathematical Analysis and Applications, 2018
- Workload characterization for microservices2016
- Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks2015
- Convex Optimization2004