Efficient Periodicity Analysis for Real-Time Anomaly Detection
Authors:
- 8 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
The analysis of time series data is critical for the effective management of IT infrastructures, with real-time anomaly detection being of particular importance. A crucial component in any anomaly detection pipeline is the characterization of individual data streams so that the most appropriate algorithm can be deployed on a case-by-case basis. Periodicity is a characteristic that directly impacts the choice of algorithm; efficient identification of its presence, or absence, is essential. In the current paper, we develop a periodicity detector based on the Lomb-Scargle periodogram and analyze its performance when exposed to varying quantities of data. Our analysis uses the well-known F1-score to identify the lower limit on the data volume required for maximal detector performance. We then extend our periodicity detection method to include a statistical analysis based on the Wilson score to establish confidence in the detector result. We demonstrate that in the absence of ground-truth labels, our method can quantity the minimum amount of data and the minimum number of trials required to correctly, and definitively, identify periodicity within a data stream.Keywords
This publication has 20 references indexed in Scilit:
- Binomial Confidence Intervals and Contingency Tests: Mathematical Fundamentals and the Evaluation of Alternative MethodsJournal of Quantitative Linguistics, 2013
- Laplace Periodogram for Time Series AnalysisJournal of the American Statistical Association, 2008
- On Periodicity Detection and Structural Periodic Similarity2005
- On the advantage of using analysis of variance for period searchMonthly Notices of the Royal Astronomical Society, 1989
- Studies in astronomical time series analysis. III - Fourier transforms, autocorrelation functions, and cross-correlation functions of unevenly spaced dataThe Astrophysical Journal, 1989
- The discrete correlation function - A new method for analyzing unevenly sampled variability dataThe Astrophysical Journal, 1988
- A period-finding method for sparse randomly spaced observations or “How long is a piece of string?”Monthly Notices of the Royal Astronomical Society, 1983
- Least-squares frequency analysis of unequally spaced dataAstrophysics and Space Science, 1976
- Periodogram Analysis and Continuous SpectraBiometrika, 1950
- Probable Inference, the Law of Succession, and Statistical InferenceJournal of the American Statistical Association, 1927