Unsupervised Anomaly Detection for Intricate KPIs via Adversarial Training of VAE
Authors:
- 1 April 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1891-1899
Abstract
To ensure the reliability of the Internet-based application services, KPIs (Key Performance Monitors) are closely monitored in real time and the anomalies presented in the KPIs must be discovered in time. While anomaly detection for the seasonal smooth service-level KPIs (e.g., number of transactions per minute) have been solved reasonably well in the literature, the intricate KPIs at the machine level (e.g., the number of I/O requests on a server monitored per second) has been little studied. These intricate KPIs are prevalent and important, but exhibit non-Gaussian noises and complex data distribution that are hard to model. In this paper, we propose an adversarial training method in the Bayesian network based on partition analysis with solid theoretical proof. Based on it, we propose the first unsupervised anomaly detection algorithmBuzz for intricate KPIs with high performance. Its best F-scores on the data from a global Internet company range from 0.92 to 0.99, significantly outperforming a state-of-art VAE-based unsupervised approach without adversarial training and a state-of-art supervised approach.Keywords
This publication has 14 references indexed in Scilit:
- Rapid and robust impact assessment of software changes in large internet-based services2015
- Opprentice2015
- Generic and Scalable Framework for Automated Time-series Anomaly Detection2015
- A provider-side view of web search response time2013
- Witnessed k-DistanceDiscrete & Computational Geometry, 2012
- Argus: End-to-end service anomaly detection and localization from an ISP's point of view2012
- ARIMA Based Network Anomaly Detection2010
- Nonparametric Estimation of Conditional DistributionsIEEE Transactions on Information Theory, 2007
- Sketch-based change detection2003
- Borel Partitions of Products of Finite Sets and the Ackermann FunctionJournal of Combinatorial Theory, Series A, 2001