HEAL: Performance Troubleshooting Deep inside Data Center Hosts
Open Access
- 10 June 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 41-42
Abstract
This study demonstrates the salient facts and challenges of host failure operations in hyperscale data centers. A host incident can involve hundreds of distinct host-level metrics. The faulting mechanism inside the host connects these heterogeneous metrics through direct and indirect correlation, making it extremely difficult to sort out the propagation procedures and the root cause from these intertwined indicators. To deeply understand the failure mechanism inside the host, we develop HEAL -- a novel host metrics analysis toolkit. HEAL discovers dynamic causality in sparse heterogeneous host metrics by combining the strengths of both time series and random variable analysis. It also extracts causal directional hints from causality's asymmetry and historical knowledge, which finally help HEAL produce accurate results given undesirable inputs. Evaluations in our production environment verify that HEAL provides significantly better result accuracy and full-process interpretability than the SOTA baselines. With these advantages, HEAL successfully serves our data center and worldwide product operations and impressively contributes to many other workflows.Keywords
Funding Information
- National Natural Science Foundation of China (62072006,92167104)
- Qiyuan Lab Innovation Fund (B0211)
- ByteDance University Research Project
This publication has 3 references indexed in Scilit:
- Causal network reconstruction from time series: From theoretical assumptions to practical estimationChaos: An Interdisciplinary Journal of Nonlinear Science, 2018
- CloudRanger: Root Cause Identification for Cloud Native Systems2018
- Improved methods of combining forecastsJournal of Forecasting, 1984