Efficient Few-Shot Classification via Contrastive Pretraining on Web Data
- 26 April 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. 4 (3) , 522-533
Abstract
Few-shot classification is challenging due to the limited data and labels. Existing algorithms usually resolve this problem by pre-training models with a considerable amount of annotated data which shares knowledge with the target domain. Nevertheless, large quantities of homogenous data samples are not always available. To tackle this obstacle, we develop a few-shot learning framework that prepares data automatically and still produces well-behaved models. This framework is implemented through conducting contrastive learning on unlabeled web images. Instead of requiring manually annotated data, this framework trains models via constructing pseudo labels. Additionally, since online data is virtually limitless and continues to be generated, the model can thus be empowered to constantly obtain up-to-date knowledge from the Internet. Furthermore, we observe that the generalization ability of learned representation is crucial for self-supervised learning. To present its importance, a naive yet efficient normalization strategy is proposed. Consequentially, this strategy boosts the accuracy of trained models significantly. We demonstrate the superiority of the proposed framework with experiments on miniImageNet, tieredImageNet and Omniglot. The results indicate that our method has surpassed previous unsupervised counterparts by a large margin and obtained performance comparable with some supervised ones.Keywords
Funding Information
- National Natural Science Foundation of China (61831014)
- Shenzhen Science and Technology Project (CJGJZD20200617102601004, ZDYBH201900000002)
This publication has 37 references indexed in Scilit:
- Colorization as a Proxy Task for Visual Understanding2017
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
- Weakly Supervised Semantic Segmentation Using Web-Crawled Videos2017
- Computational principles of synaptic memory consolidationNature Neuroscience, 2016
- Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification2016
- Deep Residual Learning for Image Recognition2016
- Machine learning: Trends, perspectives, and prospectsScience, 2015
- Stochastic Gradient Descent Tricks2012
- Learning a Similarity Metric Discriminatively, with Application to Face Verification2005
- Conceptual inadequacy of the Shannon information in quantum measurementsPhysical Review A, 2001