Concept Drift Adaptation by Exploiting Historical Knowledge
- 4 January 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 29 (10) , 4822-4832
Abstract
Incremental learning with concept drift has often been tackled by ensemble methods, where models built in the past can be retrained to attain new models for the current data. Two design questions need to be addressed in developing ensemble methods for incremental learning with concept drift, i.e., which historical (i.e., previously trained) models should be preserved and how to utilize them. A novel ensemble learning method, namely, Diversity and Transfer-based Ensemble Learning (DTEL), is proposed in this paper. Given newly arrived data, DTEL uses each preserved historical model as an initial model and further trains it with the new data via transfer learning. Furthermore, DTEL preserves a diverse set of historical models, rather than a set of historical models that are merely accurate in terms of classification accuracy. Empirical studies on 15 synthetic data streams and 5 real-world data streams (all with concept drifts) demonstrate that DTEL can handle concept drift more effectively than 4 other state-of-the-art methods.Keywords
All Related Versions
Funding Information
- Ministry of Science and Technology of China (2017YFC0804002)
- National Natural Science Foundation of China (61672478, 61329302)
- Science and Technology Innovation Committee Foundation of Shenzhen (ZDSYS201703031748284)
- Royal Society (NA150123)
This publication has 22 references indexed in Scilit:
- Resampling-Based Ensemble Methods for Online Class Imbalance LearningIEEE Transactions on Knowledge and Data Engineering, 2014
- A survey on concept drift adaptationACM Computing Surveys, 2014
- Reacting to Different Types of Concept Drift: The Accuracy Updated Ensemble AlgorithmIEEE Transactions on Neural Networks and Learning Systems, 2013
- Dynamic integration of classifiers for handling concept driftInformation Fusion, 2008
- An analysis of diversity measuresMachine Learning, 2006
- Diversity creation methods: a survey and categorisationInformation Fusion, 2005
- Learn++: an incremental learning algorithm for supervised neural networksIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2001
- Learning in the presence of concept drift and hidden contextsMachine Learning, 1996
- Incremental learning from noisy dataMachine Learning, 1986
- VII. On the association of attributes in statistics: with illustrations from the material of the childhood society, &cPhilosophical Transactions of the Royal Society A, 1900