With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations
Preprint
Abstract
Self-supervised learning algorithms based on instance discrimination train encoders to be invariant to pre-defined transformations of the same instance. While most methods treat different views of the same image as positives for a contrastive loss, we are interested in using positives from other instances in the dataset. Our method, Nearest-Neighbor Contrastive Learning of visual Representations (NNCLR), samples the nearest neighbors from the dataset in the latent space, and treats them as positives. This provides more semantic variations than pre-defined transformations. We find that using the nearest-neighbor as positive in contrastive losses improves performance significantly on ImageNet classification, from 71.7% to 75.6%, outperforming previous state-of-the-art methods. On semi-supervised learning benchmarks we improve performance significantly when only 1% ImageNet labels are available, from 53.8% to 56.5%. On transfer learning benchmarks our method outperforms state-of-the-art methods (including supervised learning with ImageNet) on 8 out of 12 downstream datasets. Furthermore, we demonstrate empirically that our method is less reliant on complex data augmentations. We see a relative reduction of only 2.1% ImageNet Top-1 accuracy when we train using only random crops.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Emerging Properties in Self-Supervised Vision Transformers2021
- Exploring Simple Siamese Representation Learning2021
- Contrastive Multiview Coding2020
- Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- 3D Object Representations for Fine-Grained Categorization2013
- Large Scale Online Learning of Image Similarity through Ranking2009