Batch Mode Active Learning for Regression With Expected Model Change
Authors:
- 18 April 2016
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 28 (7) , 1668-1681
Abstract
While active learning (AL) has been widely studied for classification problems, limited efforts have been done on AL for regression. In this paper, we introduce a new AL framework for regression, expected model change maximization (EMCM), which aims at choosing the unlabeled data instances that result in the maximum change of the current model once labeled. The model change is quantified as the difference between the current model parameters and the updated parameters after the inclusion of the newly selected examples. In light of the stochastic gradient descent learning rule, we approximate the change as the gradient of the loss function with respect to each single candidate instance. Under the EMCM framework, we propose novel AL algorithms for the linear and nonlinear regression models. In addition, by simulating the behavior of the sequential AL policy when applied for k iterations, we further extend the algorithms to batch mode AL to simultaneously choose a set of k most informative instances at each query time. Extensive experimental results on both UCI and StatLib benchmark data sets have demonstrated that the proposed algorithms are highly effective and efficient.Keywords
Funding Information
- High Technology Research and Development Program of China (2015AA015801)
- National Natural Science Foundation of China (61003107, 61521062)
- Science and Technology Commission of Shanghai Municipality (14511107500, 12DZ2272600)
This publication has 30 references indexed in Scilit:
- Active Learning of Pareto FrontsIEEE Transactions on Neural Networks and Learning Systems, 2013
- Active Learning With Drifting Streaming DataIEEE Transactions on Neural Networks and Learning Systems, 2013
- Boosted multi-task learningMachine Learning, 2010
- Pool-based active learning in approximate linear regressionMachine Learning, 2009
- Active learning for logistic regression: an evaluationMachine Learning, 2007
- Active Learning for Regression Based on Query by Committee2007
- Bootstrap prediction and Bayesian prediction under misspecified modelsBernoulli, 2005
- Greedy function approximation: A gradient boosting machine.The Annals of Statistics, 2001
- Committee-Based Sampling For Training Probabilistic Classifiers1995
- A Sequential Algorithm for Training Text Classifiers1994