TracePicker: Optimization-Based Trace Sampling for Microservice-Based Systems
Open Access
- 19 June 2025
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 2 (FSE) , 1802-1823
Abstract
Distributed tracing is a pivotal technique for software operators to understand and diagnose issues within microservice-based systems, offering a comprehensive view of user requests propagated through various services. However, the unprecedented volume of traces imposes expensive storage and analytical burdens on online systems. Conventional tracing approaches typically rely on random sampling with a fixed probability for each trace, which risks missing valuable traces. Several tail-based sampling methods have thus been proposed to sample traces based on their content. Nevertheless, these methods primarily evaluate traces on an individual basis, neglecting the collective attributes of the sample set in terms of comprehensiveness, balance, and consistency. To address these issues, we propose TracePicker, an optimization-based online sampler designed to enhance the quality of sampled data while mitigating storage burden. TracePicker employs a streaming anomaly detector to capture and retain anomalous traces that are crucial for troubleshooting. For normal traces, the sampling process is segmented into quota allocation and group sampling, both formulated as integer programming problems. By solving these problems using dynamic programming and evolution algorithms, TracePicker selects a high-quality subset of data, minimizing overall information loss. Experimental results demonstrate that TracePicker outperforms existing tail-based sampling methods in terms of both sampling quality and time consumption.Keywords
Funding Information
- the national key Research and Development Program of China (No. 2022YFF0902701)
- National Natural Science Foundation of China (No. 62032016)
- the Shenzhen Science and Technology Program (No. CJGJZD20230724091659002)
This publication has 33 references indexed in Scilit:
- Derm: SLA-aware Resource Management for Highly Dynamic Microservices2024
- Cromlech: Semi-Automated Monolith Decomposition Into MicroservicesIEEE Transactions on Services Computing, 2024
- TraceStream: Anomalous Service Localization based on Trace Stream Clustering with Online Feedback2023
- PUTraceAD: Trace Anomaly Detection with Partial Labels based on GNN and PU Learning2022
- Sieve: Attention-based Sampling of End-to-End Trace Data in Distributed Microservice Systems2021
- Practical Root Cause Localization for Microservice Systems via Trace Analysis2021
- Microscaler: Cost-Effective Scaling for Microservice Applications in the Cloud With an Online Learning ApproachIEEE Transactions on Cloud Computing, 2020
- On the usage of differential evolution for function optimization2002
- Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous SpacesJournal of Global Optimization, 1997
- The Three Sigma RuleThe American Statistician, 1994