Combating Label Noise With A General Surrogate Model For Sample Selection
Preprint
Abstract
Modern deep learning systems are data-hungry. Learning with web data is one of the feasible solutions, but will introduce label noise inevitably, which can hinder the performance of deep neural networks. Sample selection is an effective way to deal with label noise. The key is to separate clean samples based on some criterion. Previous methods pay more attention to the small loss criterion where small-loss samples are regarded as clean ones. Nevertheless, such a strategy relies on the learning dynamics of each data instance. Some noisy samples are still memorized due to frequently occurring corrupted learning patterns. To tackle this problem, a training-free surrogate model is preferred, freeing from the effect of memorization. In this work, we propose to leverage the vision-language surrogate model CLIP to filter noisy samples automatically. CLIP brings external knowledge to facilitate the selection of clean samples with its ability of text-image alignment. Furthermore, a margin adaptive loss is designed to regularize the selection bias introduced by CLIP, providing robustness to label noise. We validate the effectiveness of our proposed method on both real-world and synthetic noisy datasets. Our method achieves significant improvement without CLIP involved during the inference stage.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Co-Learning Meets Stitch-Up for Noisy Multi-Label Visual RecognitionIEEE Transactions on Image Processing, 2023
- Instance-Dependent Label-Noise Learning with Manifold-Regularized Transition Matrix Estimation2022
- Weakly Supervised Moment Localization with Decoupled Consistent Concept PredictionInternational Journal of Computer Vision, 2022
- Multiple knowledge representation for big data artificial intelligence: framework, applications, and case studiesFrontiers of Information Technology & Electronic Engineering, 2021
- Understanding deep learning (still) requires rethinking generalizationCommunications of the ACM, 2021
- Big Transfer (BiT): General Visual Representation Learning2020
- Deep Learning for Generic Object Detection: A SurveyInternational Journal of Computer Vision, 2019