Open Access
- 5 July 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 666-677
Abstract
Maintaining up-to-date troubleshooting guides (TSGs) is critical for the reliability of cloud systems, yet manual maintenance often leads to inefficiencies and outdated documentation. This paper proposes TSGen, an automated pipeline for generating high-quality, structured TSGs from historical incident reports using large language models (LLMs). Our approach consists of three stages: (1) filtering and classifying incident data into diagnostically relevant categories, (2) distilling core incidents to ensure diversity and generalizability, and (3) organizing the distilled knowledge into a directed acyclic graph (DAG) that captures root causes and resolutions in a structured manner. By leveraging real-world incident discussions, TSGen produces dynamic and reusable guides tailored for live troubleshooting. Experiments on real-world incidents from Microsoft demonstrate that TSGen achieves 54.8% incident coverage and approximately 3× higher retrieval accuracy compared to baselines. Furthermore, the system supports iterative updates, allowing guides to evolve alongside dynamic cloud environments. Human evaluation shows that on-call engineers rate these generated TSGs significantly higher than human-crafted ones.Keywords
This publication has 3 references indexed in Scilit:
- Ward’s Hierarchical Agglomerative Clustering Method: Which Algorithms Implement Ward’s Criterion?Journal of Classification, 2014
- Term-weighting approaches in automatic text retrievalInformation Processing & Management, 1988
- Hierarchical Grouping to Optimize an Objective FunctionJournal of the American Statistical Association, 1963