Survey on Coresets for Deep Learning: Methods and Applications
Preprint
- 11 September 2026
- preprint
- Published by MDPI AG in Preprints.org
Abstract
This survey presents a comprehensive review of coreset methods in deep learning, an important tool for improving data efficiency in large-scale neural networks. In general, “coreset” is an algorithmic technique for selecting a small yet representative subset of data to replace the full dataset, which can yield more efficient training process and meanwhile preserve model performance. In the past 20 years, coreset techniques have been widely applied to many classical machine learning problems, such as clustering, regression and classification. In recent years, the coreset techniques also begin to attract a lot of attention in modern deep learning area. However, designing effective coresets usually is a challenging task since we need to take account of the trade-off among multiple different factors, such as complexity, robustness and accuracy. In this survey, we focus on two common scenarios for using coreset methods in deep learning: (1) reducing the extremely high computational cost for training a deep learning model, and (2) improving the data utilization under resource constraints such as limited label budget or storage capacity. We begin by outlining the fundamental principles, advantages, and design challenges of coresets for these two scenarios. We also discuss the emerging applications of coresets in large language models. Finally, we identify several open problems and promising directions for future research.Keywords
All Related Versions
This publication has 75 references indexed in Scilit:
- Adversarial Coreset Selection for Efficient Robust TrainingInternational Journal of Computer Vision, 2023
- Deep active sampling with self-supervised learningFrontiers of Computer Science, 2022
- Core-Sets: Updated Survey2019
- Coresets-Methods and History: A Theoreticians Design Pattern for Approximation and Streaming AlgorithmsKI - Künstliche Intelligenz, 2017
- Multi-label active learning by model guided distribution matchingFrontiers of Computer Science, 2016
- Monte Carlo Statistical Methods2004
- Principal components analysis (PCA)Computers & Geosciences, 1993
- Clustering to minimize the maximum intercluster distanceTheoretical Computer Science, 1985
- The Fréchet distance between multivariate normal distributionsJournal of Multivariate Analysis, 1982
- An analysis of approximations for maximizing submodular set functions—IMathematical Programming, 1978