Colorization as a Proxy Task for Visual Understanding
Preprint
Abstract
We investigate and improve self-supervision as a drop-in replacement for ImageNet pretraining, focusing on automatic colorization as the proxy task. Self-supervised training has been shown to be more promising for utilizing unlabeled data than other, traditional unsupervised learning methods. We show the ability of our self-supervised network in several contexts. On VOC segmentation and classification tasks, we present results that are state-of-the-art among methods not using ImageNet labels for pretraining representations. Moreover, we present the first in-depth analysis of self-supervision via colorization, concluding that formulation of the loss, training details and network architecture play important roles in its effectiveness. This investigation is further expanded by revisiting the ImageNet pretraining paradigm, asking questions such as: How much training data is needed? How many labels are needed? How much do features change when fine-tuned? We relate these questions back to self-supervision by showing that self-supervised colorization provides a similarly powerful supervision as various flavors of ImageNet pretraining.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles2016
- Learning Representations for Automatic Colorization2016
- What’s the Point: Semantic Segmentation with Point Supervision2016
- Context Encoders: Feature Learning by Inpainting2016
- Feedforward semantic segmentation with zoom-out features2015
- Hypercolumns for object segmentation and fine-grained localization2015
- Learning to segment under various forms of weak supervision2015
- Reconstructive Sparse Code Transfer for Contour Detection and Semantic Labeling2015
- On the momentum term in gradient descent learning algorithmsNeural Networks, 1999
- Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortexProceedings Of The Royal Society B-Biological Sciences, 1998