Concept Drift Adaptation by Exploiting Historical Knowledge
Preprint
Abstract
Incremental learning with concept drift has often been tackled by ensemble methods, where models built in the past can be re-trained to attain new models for the current data. Two design questions need to be addressed in developing ensemble methods for incremental learning with concept drift, i.e., which historical (i.e., previously trained) models should be preserved and how to utilize them. A novel ensemble learning method, namely Diversity and Transfer based Ensemble Learning (DTEL), is proposed in this paper. Given newly arrived data, DTEL uses each preserved historical model as an initial model and further trains it with the new data via transfer learning. Furthermore, DTEL preserves a diverse set of historical models, rather than a set of historical models that are merely accurate in terms of classification accuracy. Empirical studies on 15 synthetic data streams and 4 real-world data streams (all with concept drifts) demonstrate that DTEL can handle concept drift more effectively than 4 other state-of-the-art methods.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Online Ensemble Learning of Data Streams with Gradually Evolved ClassesIEEE Transactions on Knowledge and Data Engineering, 2016
- Resampling-Based Ensemble Methods for Online Class Imbalance LearningIEEE Transactions on Knowledge and Data Engineering, 2014
- Reacting to Different Types of Concept Drift: The Accuracy Updated Ensemble AlgorithmIEEE Transactions on Neural Networks and Learning Systems, 2013
- Incremental Learning From Stream DataIEEE Transactions on Neural Networks, 2011
- Incremental Learning of Concept Drift in Nonstationary EnvironmentsIEEE Transactions on Neural Networks, 2011
- DDD: A New Ensemble Approach for Dealing with Concept DriftIEEE Transactions on Knowledge and Data Engineering, 2011
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009
- The Impact of Diversity on Online Ensemble Learning in the Presence of Concept DriftIEEE Transactions on Knowledge and Data Engineering, 2009
- Dynamic integration of classifiers for handling concept driftInformation Fusion, 2008
- Learn++: an incremental learning algorithm for supervised neural networksIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2001