Sieve: Attention-based Sampling of End-to-End Trace Data in Distributed Microservice Systems
Authors:
- 1 September 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 436-446
Abstract
End-to-end tracing plays an important role in understanding and monitoring distributed microservice systems. The trace data are valuable to help find out the anomalous or erroneous behavior of the system. However, the volume of trace data is huge leading to a heavy burden on analyzing and storing them. To reduce the volume of trace data, the sampling technique is widely adopted. However, existing uniform sampling approaches are unable to capture uncommon traces that are more interesting and informative. To tackle this problem, we design and implement Sieve, an online sampler that aims to bias sampling towards uncommon traces by taking advantage of the attention mechanism. The evaluation results on the trace datasets collected from real-world and experimental microservice systems show that Sieve is effective to increase sampling probabilities of the structurally and temporally uncommon traces and reduce the storage space to a large extent by taking a low sampling rate.Keywords
This publication has 17 references indexed in Scilit:
- Microscope: Pinpoint Performance Issues with Causal Graphs in Micro-service Environments2018
- Weighted Sampling of Execution Traces2018
- Overload Control for Scaling WeChat Microservices2018
- Delta debugging microservice systems2018
- SiPTA2014
- Error-bounded sampling for analytics on big sparse dataProceedings of the VLDB Endowment, 2014
- Draco: Statistical diagnosis of chronic problems in large distributed systems2012
- Isolation-Based Anomaly DetectionACM Transactions on Knowledge Discovery From Data, 2012
- Stardust2006
- Pinpoint: problem determination in large, dynamic Internet services2003