A Robust Data‐Driven Method for Multiseasonality and Heteroscedasticity in Time Series Preprocessing
Open Access
- 1 January 2021
- journal article
- research article
- Published by Wiley in Wireless Communications and Mobile Computing
- Vol. 2021 (1)
Abstract
Internet of Things (IoT) is emerging, and 5G enables much more data transport from mobile and wireless sources. The data to be transmitted is too much compared to link capacity. Labelling data and transmit only useful part of the collected data or their features is a promising solution for this challenge. Abnormal data are valuable due to the need to train models and to detect anomalies when being compared to already overflowing normal data. Labelling can be done in data sources or edges to balance the load and computing between sources, edges, and centres. However, unsupervised labelling method is still a challenge preventing to implement the above solutions. Two main problems in unsupervised labelling are long‐term dynamic multiseasonality and heteroscedasticity. This paper proposes a data‐driven method to handle modelling and heteroscedasticity problems. The method contains the following main steps. First, raw data are preprocessed and grouped. Second, main models are built for each group. Third, models are adapted back to the original measured data to get raw residuals. Fourth, raw residuals go through deheteroscedasticity and become normalized residuals. Finally, normalized residuals are used to conduct anomaly detection. The experimental results with real‐world data show that our method successfully increases receiver‐operating characteristic (AUC) by about 30%.Keywords
Funding Information
- Natural Science Foundation of Shandong Province (ZR2020MF067)
This publication has 28 references indexed in Scilit:
- Adaptive robust INS/UWB-integrated human tracking using UFIR filter bankMeasurement, 2018
- Short‐term traffic forecasting using self‐adjusting k‐nearest neighboursIET Intelligent Transport Systems, 2017
- Unsupervised real-time anomaly detection for streaming dataNeurocomputing, 2017
- Correcting and complementing freeway traffic accident data using Mahalanobis distance based outlier detectionTehnicki vjesnik - Technical Gazette, 2017
- Unsupervised outlier detection for time series by entropy and dynamic time warpingKnowledge and Information Systems, 2017
- Detection of Network Attacks Using Hybrid ARIMA-GARCH Model2017
- KDE based outlier detection on distributed data streams in multimedia networkMultimedia Tools and Applications, 2016
- Online active learning of decision trees with evidential dataPattern Recognition, 2016
- Propagating Disaster Warnings on Social and Digital Media2015
- Constrained M-Estimation for Regression1996