Probabilistic Temporal Fusion Transformers for Large-Scale KPI Anomaly Detection
Open Access
- 12 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 9123-9137
Abstract
This paper introduces a new generic and scalable framework for large-scale time series prediction and unsupervised anomaly detection. The most common approach of state-of-the-art time series anomaly detection techniques, which are mostly based on neural networks, is to train a network per time series. However, a typical modern microservice system consists of hundreds of active nodes/instances. To monitor the performance of such a system, we often need to keep track of thousands of time series describing different aspects of the system, including CPU usage, call latency, and workloads. We introduce a new methodology for grouping metrics that share the same type, predicting hundreds of metrics concurrently with a single neural network model with shared parameters. The model also integrates the probabilistic representations and Temporal Fusion Transformers for better performance. In a real-world dataset, our proposed model achieved up to 50% improvement in terms of MSE.Keywords
Funding Information
- China Telecom Company
This publication has 29 references indexed in Scilit:
- Temporal Fusion Transformers for interpretable multi-horizon time series forecastingInternational Journal of Forecasting, 2021
- Unsupervised Detection of Microservice Trace Anomalies through Service-Level Deep Bayesian Networks2020
- DeepAR: Probabilistic forecasting with autoregressive recurrent networksInternational Journal of Forecasting, 2020
- DSTP-RNN: A dual-stage two-phase attention-based recurrent neural network for long-term and multivariate time series predictionExpert Systems with Applications, 2020
- Energy Disaggregation via Deep Temporal Dictionary LearningIEEE Transactions on Neural Networks and Learning Systems, 2019
- An RBM Anomaly Detector for the Cloud2019
- Rough Deep Neural Architecture for Short-Term Wind Speed ForecastingIEEE Transactions on Industrial Informatics, 2017
- Large-Scale Unusual Time Series Detection2015
- Time-series clustering – A decade reviewInformation Systems, 2015
- Workload-Aware Online Anomaly Detection in Enterprise Applications with Local Outlier Factor2012