Combating Noisy Labels by Alleviating the Memorization of DNNs to Noisy Labels
- 24 December 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 597-609
Abstract
Data is the essential fuel for deep neural networks (DNNs), and its quality affects the practical performance of DNNs. In real-world training scenarios, the successful generalization performance of DNNs is severely challenged by noisy samples with incorrect labels. To combat noisy samples in image classification, numerous methods based on sample selection and semi-supervised learning (SSL) have been developed, where sample selection is used to provide the supervision signal for SSL, achieving great success in resisting noisy samples. Due to the necessary warm-up training on noisy datasets and the basic sample selection mechanism, DNNs are still confronted with the challenge of memorizing noisy samples. However, existing methods do not address the memorization of noisy samples by DNNs explicitly, which hinders the generalization performance of DNNs. To alleviate this issue, we present a new approach to combat noisy samples. First, we propose a memorized noise detection method to detect noisy samples that DNNs have already memorized during the training process. Next, we design a noise-excluded sample selection method and a noise-alleviated MixMatch to alleviate the memorization of DNNs to noisy samples. Finally, we integrate our approach with the established method DivideMix, proposing Modified-DivideMix. The experimental results on CIFAR-10, CIFAR-100, and Clothing1M demonstrate the effectiveness of our approachKeywords
Funding Information
- National Natural Science Foundation of China (62125205, U23A20303)
- Key Research and Development Projects of Shaanxi Province (2021ZDLGYO5-04)
This publication has 40 references indexed in Scilit:
- Neighborhood Collective Estimation for Noisy Label Identification and Correction2022
- ConMatch: Semi-supervised Learning with Confidence-Guided Consistency Regularization2022
- ScanMix: Learning from Severe Label Noise via Semantic Clustering and Semi-Supervised LearningPattern Recognition, 2022
- LongReMix: Robust learning with high confidence samples in a noisy label environmentPattern Recognition, 2022
- Data poisoning attacks against machine learning algorithmsExpert Systems with Applications, 2022
- Label Sanitization Against Label Flipping Poisoning Attacks2019
- Retrospective Encoders for Video Summarization2018
- Learning with Biased Complementary Labels2018
- Learning from multiple annotators with varying expertiseMachine Learning, 2013
- A study of Gaussian mixture models of color and texture features for image classification and segmentationPattern Recognition, 2006