Microscaler: Cost-Effective Scaling for Microservice Applications in the Cloud With an Online Learning Approach
- 6 April 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cloud Computing
- Vol. 10 (2) , 1100-1116
Abstract
Recently, the microservice becomes a popular architecture to construct cloud native systems due to its agility. In cloud native systems, autoscaling is a key enabling technique to adapt to workload changes by acquiring or releasing the right amount of computing resources. However, it becomes a challenging problem in microservice applications, since such an application usually comprises a large number of different microservices with complex interactions. When the performance decreases due to an unpredictable workload peak, it is difficult to pinpoint the scaling-needed services which need to scale out and evaluate how many resources they need. In this article, we present a novel system named Microscaler to automatically identify the scaling-needed services and scale them to meet the Service Level Agreement (SLA) with an optimal cost for microservice applications. Microscaler first collects the quality of service (QoS) metrics in the service mesh enabled microservice infrastructure. Then, it determines under-provisioning or over-provisioning service instances along the service dependency graph with a novel scaling-needed service criterion named service power . The service dependency graph could be obtained by correlating each request flow in the service mesh. By combining an online learning approach and a step-by-step heuristic approach, Microscaler can precisely reach the optimal service scale meeting the SLA requirements. The experimental evaluations in a microservice benchmark show that Microscaler achieves an average 93 percent precision in scaling-needed service determination and converges to the optimal service scale faster than several state-of-the-art methods. Moreover, Microscaler is lightweight and flexible enough to work in a large-scale microservice system.Keywords
Funding Information
- National Key R&D Program of China (2019YFB1804002)
- National Natural Science Foundation of China (61802448, 61722214, U1811462)
- National Science Foundation of Guangdong (2019A1515012229)
This publication has 40 references indexed in Scilit:
- Human-level control through deep reinforcement learningNature, 2015
- On Cloud Service Reliability Enhancement with Optimal Resource UsageIEEE Transactions on Cloud Computing, 2014
- Aggressive Resource Provisioning for Ensuring QoS in Virtualized EnvironmentsIEEE Transactions on Cloud Computing, 2014
- A cost-aware auto-scaling approach using the workload prediction in service cloudsInformation Systems Frontiers, 2013
- AutoScaleACM Transactions on Computer Systems, 2012
- Applying reinforcement learning towards automating resource allocation and application scalability in the cloudConcurrency and Computation: Practice and Experience, 2012
- Elastic Virtual Machine for Fine-Grained Cloud Resource Provisioning2012
- Empirical prediction models for adaptive resource provisioning in the cloudFuture Generation Computer Systems, 2012
- A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of NoiseJournal of Basic Engineering, 1964
- Unbiased Estimation of Certain Correlation CoefficientsThe Annals of Mathematical Statistics, 1958