RCAEval: A Benchmark for Root Cause Analysis of Microservice Systems with Telemetry Data
Authors:
Preprint
Abstract
Root cause analysis (RCA) for microservice systems has gained significant attention in recent years. However, there is still no standard benchmark that includes large-scale datasets and supports comprehensive evaluation environments. In this paper, we introduce RCAEval, an open-source benchmark that provides datasets and an evaluation environment for RCA in microservice systems. First, we introduce three comprehensive datasets comprising 735 failure cases collected from three microservice systems, covering various fault types observed in real-world failures. Second, we present a comprehensive evaluation framework that includes fifteen reproducible baselines covering a wide range of RCA approaches, with the ability to evaluate both coarse-grained and fine-grained RCA. RCAEval is designed to support both researchers and practitioners. We hope that this ready-to-use benchmark will enable researchers and practitioners to conduct extensive analysis and pave the way for robust new solutions for RCA of microservice systems.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Root Cause Analysis for Microservice System based on Causal Inference: How Far Are We?2024
- BARO: Robust Root Cause Analysis for Microservices via Multivariate Bayesian Online Change Point DetectionProceedings of the ACM on Software Engineering, 2024
- Nezha: Interpretable Fine-Grained Root Causes Analysis for Microservices on Multi-modal Observability Data2023
- Robust Failure Diagnosis of Microservice System Through Multimodal DataIEEE Transactions on Services Computing, 2023
- Eadro: An End-to-End Troubleshooting Framework for Microservices on Multi-source Data2023
- Causal Inference-Based Root Cause Analysis for Online Service Systems with Intervention Recognition2022
- Diagnosing Performance Issues in Microservices with Heterogeneous Data Source2021
- Practical Root Cause Localization for Microservice Systems via Trace Analysis2021
- MicroRank: End-to-End Latency Issue Localization with Extended Spectrum Analysis in Microservice Environments2021
- How bad can a bug get? an empirical analysis of software failures in the OpenStack cloud computing platform2019