Transitive Invariance for Self-Supervised Visual Representation Learning
Authors:
- Xiaolong WangXWXiaolong Wang
- Kaiming HeKHKaiming He
- Abhinav GuptaAGAbhinav Gupta
- 1 October 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1338-1347
Abstract
Learning visual representations with self-supervised learning has become popular in computer vision. The idea is to design auxiliary tasks where labels are free to obtain. Most of these tasks end up providing data to learn specific kinds of invariance useful for recognition. In this paper, we propose to exploit different self-supervised approaches to learn representations invariant to (i) inter-instance variations (two objects in the same class should have similar features) and (ii) intra-instance variations (viewpoint, pose, deformations, illumination, etc.). Instead of combining two approaches with multi-task learning, we argue to organize and reason the data with multiple variations. Specifically, we propose to generate a graph with millions of objects mined from hundreds of thousands of videos. The objects are connected by two types of edges which correspond to two types of invariance: “different instances but a similar viewpoint and category” and “different viewpoints of the same instance”. By applying simple transitivity on the graph with these edges, we can obtain pairs of images exhibiting richer visual invariance. We use this data to train a Triplet-Siamese network with VGG16 as the base architecture and apply the learned representations to different recognition tasks. For object detection, we achieve 63.2% mAP on PASCAL VOC 2007 using Fast R-CNN (compare to 67.3% with ImageNet pre-training). For the challenging COCO dataset, our method is surprisingly close (23.5%) to the ImageNet-supervised counterpart (24.4%) using the Faster R-CNN framework. We also show that our network can perform significantly better than the ImageNet network in the surface normal estimation task.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- High-Speed Tracking with Kernelized Correlation FiltersIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Selective Search for Object RecognitionInternational Journal of Computer Vision, 2013
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- Using Multiple Segmentations to Discover Objects and their Extent in Image Collections2006
- Learning a Similarity Metric Discriminatively, with Application to Face Verification2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Sparse coding with an overcomplete basis set: A strategy employed by V1?Vision Research, 1997
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989