On Learning Contrastive Representations for Learning with Noisy Labels
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 16661-16670
Abstract
Deep neural networks are able to memorize noisy labels easily with a softmax cross entropy (CE) loss. Previous studies attempted to address this issue focus on incorporating a noise-robust loss function to the CE loss. However, the memorization issue is alleviated but still remains due to the non-robust CE loss. To address this issue, we focus on learning robust contrastive representations of data on which the classifier is hard to memorize the label noise under the CE loss. We propose a novel contrastive regularization function to learn such representations over noisy data where label noise does not dominate the representation learning. By theoretically investigating the representations induced by the proposed regularization function, we reveal that the learned representations keep information related to true labels and discard information related to corrupted labels. Moreover, our theoretical results also indicate that the learned representations are robust to the label noise. The effectiveness of this method is demonstrated with experiments on benchmark datasets.Keywords
All Related Versions
Funding Information
- NSERC (DMS 2009752)
This publication has 20 references indexed in Scilit:
- Symmetric Cross Entropy for Robust Learning With Noisy Labels2019
- Probabilistic End-To-End Noise Correction for Learning With Noisy Labels2019
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization2017
- Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach2017
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Deep Residual Learning for Image Recognition2016
- Learning from massive noisy labeled data for image classification2015
- Noise Tolerance Under Risk MinimizationIEEE Transactions on Cybernetics, 2013
- A theory of learning from different domainsMachine Learning, 2009
- Analysis of Representations for Domain Adaptation2007