STEAM: Observability-Preserving Trace Sampling
Open Access
- 30 November 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1750-1761
Abstract
In distributed systems and microservice applications, tracing is a crucial observability signal employed for comprehending their internal states. To mitigate the overhead associated with distributed tracing, most tracing frameworks utilize a uniform sampling strategy, which retains only a subset of traces. However, this approach is insufficient for preserving system observability. This is primarily attributed to the long-tail distribution of traces in practice, which results in the omission or rarity of minority yet critical traces after sampling. In this study, we introduce an observability-preserving trace sampling method, denoted as STEAM, which aims to retain as much information as possible in the sampled traces. We employ Graph Neural Networks (GNN) for trace representation, while incorporating domain knowledge of trace comparison through logical clauses. Subsequently, we employ a scalable approach to sample traces, emphasizing mutually dissimilar traces. STEAM has been implemented on top of OpenTelemetry, comprising approximately 1.6K lines of Golang code and 2K lines of Python code. Evaluation on four benchmark microservice applications and a production system demonstrates the superior performance of our approach compared to baseline methods. Furthermore, STEAM is capable of processing 15,000 traces in approximately 4 seconds.Keywords
This publication has 17 references indexed in Scilit:
- Sifter2019
- Latent error prediction and fault localization for microservice applications by learning from system trace logs2019
- An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud & Edge Systems2019
- Fault Analysis and Debugging of Microservice Systems: Industrial Survey, Benchmark System, and Empirical StudyIEEE Transactions on Software Engineering, 2018
- Weighted Sampling of Execution Traces2018
- Canopy2017
- A Hierarchical Algorithm for Extreme Clustering2017
- Pivot tracing2015
- Determinantal Point Processes for Machine Learning2012
- On the general theory of control systemsIFAC Proceedings Volumes, 1960