Unsupervised Bayesian Nonparametric Approach with Incremental Similarity Tracking of Unlabeled Water Demand Time Series for Anomaly Detection
Open Access
Abstract
In this paper, a fusion of unsupervised clustering and incremental similarity tracking of hourly water demand series is proposed. Current research using unsupervised methodologies to detect anomalous water is limited and may possess several limitations such as a large amount of dataset, the need to select an optimal cluster number, or low detection accuracy. Our proposed approach aims to address the need for a large amount of dataset by detecting anomaly through (1) clustering points that are relatively similar at each time step, (2) clustering points at each time step by the similarity in how they vary from each time step, and (3) to compare the incoming points with a reference shape for online anomalous trend detection. Secondly, through the use of Bayesian nonparametric approach such as the Dirichlet Process Mixture Model, the need to choose an optimal cluster number is eliminated and provides a subtle solution for ‘reserving’ an empty cluster for the future anomaly. Among the 165 randomly generated anomalies, the proposed approach detected a total of 159 anomalies and other anomalous trends present in the data. As the data is unlabeled, identified anomalous trends cannot be verified. However, results show great potential in using minimally unlabeled water demand data for a preliminary anomaly detection.Keywords
Funding Information
- Economic Development Board - Singapore (BH180750)
This publication has 26 references indexed in Scilit:
- Water consumption patterns as a basis for water demand modelingWater Resources Research, 2015
- Adaptive water demand forecasting for near real-time management of smart water distribution systemsEnvironmental Modelling & Software, 2014
- Weighted Least Squares with Expectation-Maximization Algorithm for Burst Detection in U.K. Water Distribution SystemsJournal of Water Resources Planning and Management, 2014
- Sensing-Transmission Edifice Using Bayesian Nonparametric Traffic Clustering in Cognitive Radio NetworksIEEE Transactions on Mobile Computing, 2013
- ANN-based residential water end-use demand forecasting modelExpert Systems with Applications, 2013
- Principal Factor Analysis for Forecasting Diurnal Water-Demand Pattern Using Combined Rough-Set and Fuzzy-Clustering TechniqueJournal of Water Resources Planning and Management, 2013
- A tutorial on Bayesian nonparametric modelsJournal of Mathematical Psychology, 2012
- A novel clustering method on time series dataExpert Systems with Applications, 2011
- Novelty detection for time series data analysis in water distribution systems using support vector machinesJournal of Hydroinformatics, 2010
- Predictive models for forecasting hourly urban water demandJournal of Hydrology, 2010