Data Shapley Valuation for Efficient Batch Active Learning
Authors:
- 31 October 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1456-1462
Abstract
Annotating the right set of data amongst all available data points is a key challenge in many machine learning applications. Batch active learning is a popular approach to address this, in which batches of unlabeled data points are selected for annotation, while an underlying learning algorithm gets subsequently updated. In this work, we introduce Active Data Shapley (ADS) –a filtering layer for batch active learning that significantly increases the efficiency of existing active learning algorithms by pre-selecting, using a linear time computation, the highest-value points from an unlabeled dataset. Using the notion of the Shapley value of data, our method estimates the value of unlabeled data points with regards to the prediction task at hand. We show that ADS is particularly effective when the pool of unlabeled data exhibits real-world caveats: noise, heterogeneity, and domain shift. We run experiments demonstrating that when ADS is used to pre-select the highest-ranking portion of an unlabeled dataset, the efficiency of state-of-the-art batch active learning methods increases by an average factor of 6x, while preserving performance effectiveness.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Exploring Representativeness and Informativeness for Active LearningIEEE Transactions on Cybernetics, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Active learning for sparse bayesian multilabel classification2014
- Clustering Based Active Learning for Evolving Data Streams2013
- Active Learning for Hierarchical Text ClassificationLecture Notes in Computer Science, 2012
- Active learning in multimedia annotation and retrievalACM Transactions on Intelligent Systems and Technology, 2011
- Smaller Coresets for k-Median and k-Means ClusteringDiscrete & Computational Geometry, 2006
- Batch mode active learning and its application to medical image classification2006
- Diverse ensembles for active learning2004
- Active Hidden Markov Models for Information Extraction2001