Interpolation-Based Contrastive Learning for Few-Label Semi-Supervised Learning
- 7 July 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (2) , 2054-2065
Abstract
Semi-supervised learning (SSL) has long been proved to be an effective technique to construct powerful models with limited labels. In the existing literature, consistency regularization-based methods, which force the perturbed samples to have similar predictions with the original ones have attracted much attention for their promising accuracy. However, we observe that the performance of such methods decreases drastically when the labels get extremely limited, e.g., $2$ or $3$ labels for each category. Our empirical study finds that the main problem lies with the drift of semantic information in the procedure of data augmentation. The problem can be alleviated when enough supervision is provided. However, when little guidance is available, the incorrect regularization would mislead the network and undermine the performance of the algorithm. To tackle the problem, we: 1) propose an interpolation-based method to construct more reliable positive sample pairs and 2) design a novel contrastive loss to guide the embedding of the learned network to change linearly between samples so as to improve the discriminative capability of the network by enlarging the margin decision boundaries. Since no destructive regularization is introduced, the performance of our proposed algorithm is largely improved. Specifically, the proposed algorithm outperforms the second best algorithm (Comatch) with $5.3\%$ by achieving $88.73\%$ classification accuracy when only two labels are available for each class on the CIFAR-10 dataset. Moreover, we further prove the generality of the proposed method by improving the performance of the existing state-of-the-art algorithms considerably with our proposed strategy. The corresponding code is available at https://github.com/xihongyang1999/ICL_SSL.
Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2020AAA0107100)
- National Natural Science Foundation of China (62006237, 61825305, 61922088, 61906020, 61773392)
This publication has 33 references indexed in Scilit:
- Adaptive Semi-Supervised Feature Selection for Cross-Modal RetrievalIEEE Transactions on Multimedia, 2018
- Exploring the Limits of Weakly Supervised Pretraining2018
- Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised LearningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Between-Class Learning for Image Classification2018
- Unsupervised Feature Learning via Non-parametric Instance Discrimination2018
- Robust Adaptive Embedded Label Propagation With Weight Learning for Inductive ClassificationIEEE Transactions on Neural Networks and Learning Systems, 2017
- Can the Virtual Labels Obtained by Traditional LP Approaches Be Well Encoded in WLR?IEEE Transactions on Neural Networks and Learning Systems, 2015
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- Trace Ratio Optimization-Based Semi-Supervised Nonlinear Dimensionality Reduction for Marginal Manifold VisualizationIEEE Transactions on Knowledge and Data Engineering, 2012
- Target Detection and Localization Using MIMO Radars and SonarsIEEE Transactions on Signal Processing, 2006