Regularly Truncated M-Estimators for Learning With Noisy Labels
- 28 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (5) , 3522-3536
Abstract
The sample selection approach is very popular in learning with noisy labels. As deep networks “learn pattern first” , prior methods built on sample selection share a similar training procedure: the small-loss examples can be regarded as clean examples and used for helping generalization, while the large-loss examples are treated as mislabeled ones and excluded from network parameter updates. However, such a procedure is arguably debatable from two folds: (a) it does not consider the bad influence of noisy labels in selected small-loss examples; (b) it does not make good use of the discarded large-loss examples, which may be clean or have meaningful information for generalization. In this paper, we propose regularly truncated M-estimators (RTME) to address the above two issues simultaneously . Specifically, RTME can alternately switch modes between truncated M-estimators and original M-estimators . The former can adaptively select small-losses examples without knowing the noise rate and reduce the side-effects of noisy labels in them. The latter makes the possibly clean examples but with large losses involved to help generalization. Theoretically, we demonstrate that our strategies are label-noise-tolerant. Empirically, comprehensive experimental results show that our method can outperform multiple baselines and is robust to broad noise types and levels. The implementation is available at https://github.com/xiaoboxia/RTM_LNL .Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62336003, 12371510)
- NSF of Jiangsu Province (BZ2021013, BK20220080)
- NSFC Young Scientists Fund (62006202)
- NSFC General Program (62376235)
- RGC Early Career Scheme (22200720)
- National Natural Science Foundation of China (62276242)
- National Aviation Science Foundation (2022Z071078001)
- CAAI-Huawei MindSpore Open Fund (CAAIXSJLJJ-2021-016B, CAAIXSJLJJ-2022-001A)
- Anhui Province Key Research and Development Program (202104a05020007)
- DreamsFoundation of Jianghuai Advance Technology Center (2023-ZM01Z001)
- USTC-IAT Application Sci. & Tech. Achievement Cultivation Program (JL06521001Y)
- Sci. & Tech. Innovation Special Zone (20-163-14-LZ-001-004-01)
- National Natural Science Foundation of China (62125201, 62020106007)
- Australian Research Council (FT220100318, DP220102121, LP220100527, LP220200949, IC190100031)
This publication has 50 references indexed in Scilit:
- Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection2022
- Probability2019
- A brief introduction to weakly supervised learningNational Science Review, 2017
- Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach2017
- Empirical Risk Minimization2017
- Learning from massive noisy labeled data for image classification2015
- Food-101 – Mining Discriminative Components with Random Forests2014
- An Introduction to Statistical Learning2013
- Enhancing Sparsity by Reweighted ℓ 1 MinimizationJournal of Fourier Analysis and Applications, 2008
- NewsWeeder: Learning to Filter Netnews1995