Concept drift detector based on centroid distance analysis
- 18 July 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
The interest in data stream mining is continuously growing due to the increasing volume of data arriving at high speed produced by various systems. Processing and classification to gain valuable information from this kind of data require special treatment. The main problem is the non-stationary nature of data streams, which reveals in concept drifts - changes in sample characteristics describing classes. There are many approaches to mitigate the adverse effects of this phenomenon. However, one of the most popular techniques is drift detection. This work will propose a new drift detection approach based on the distance analysis between the subsequent data chunk centroids. The original solution in this methodology is to gather and preserve information about the data characteristics starting from the previous drift detection or data stream beginning. Two variants of the proposed method were compared with state-of-the-art concept drift detection algorithms. The evaluation was made on a comprehensive set of synthetic streams with different concept drift types and selected real streams. The evaluated results subjected to statistical analysis showed that the proposed concept drift detection algorithm could improve the classification of tested classifiers with a statistically significant advantage over the compared drift detection methods.Keywords
This publication has 22 references indexed in Scilit:
- Lessons learned from data stream classification applied to credit scoringExpert Systems with Applications, 2020
- An overview of unsupervised drift detection methodsWIREs Data Mining and Knowledge Discovery, 2020
- Challenges in benchmarking stream learning algorithms with real-world dataData Mining and Knowledge Discovery, 2020
- Data stream classification: a reviewIran Journal of Computer Science, 2020
- A dynamic financial distress forecast model with multiple forecast results under unbalanced data environmentKnowledge-Based Systems, 2020
- A large-scale comparison of concept drift detectorsInformation Sciences, 2018
- Accumulating regional density dissimilarity for concept drift detection in data streamsPattern Recognition, 2018
- Semi Supervised Adaptive Framework for Classifying Evolving Data Stream2015
- Advances in data stream miningWIREs Data Mining and Knowledge Discovery, 2011
- Detecting Concept Drift Using Statistical Testing2007