GAN-Based Image Colorization for Self-Supervised Visual Feature Learning
Open Access
Abstract
Large-scale labeled datasets are generally necessary for successfully training a deep neural network in the computer vision domain. In order to avoid the costly and tedious work of manually annotating image datasets, self-supervised learning methods have been proposed to learn general visual features automatically. In this paper, we first focus on image colorization with generative adversarial networks (GANs) because of their ability to generate the most realistic colorization results. Then, via transfer learning, we use this as a proxy task for visual understanding. Particularly, we propose to use conditional GANs (cGANs) for image colorization and transfer the gained knowledge to two other downstream tasks, namely, multilabel image classification and semantic segmentation. This is the first time that GANs have been used for self-supervised feature learning through image colorization. Through extensive experiments with the COCO and Pascal datasets, we show an increase of 5% for the classification task and 2.5% for the segmentation task. This demonstrates that image colorization with conditional GANs can boost other downstream tasks’ performance without the need for manual annotation.Keywords
Funding Information
- FEDER---PT2020 partnership agreement (UIDB/50008/2020)
- FCT/MEC through national funds (FCT/MEC)
This publication has 45 references indexed in Scilit:
- Learning Representations for Automatic Colorization2016
- Let there be color!ACM Transactions on Graphics, 2016
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Learning to See by Moving2015
- Learning Large-Scale Automatic Image Colorization2015
- Fast R-CNN2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- Multi-Label ClassificationInternational Journal of Data Warehousing and Mining, 2007
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004