Semantic‐aware automatic image colorization via unpaired cycle‐consistent self‐supervised network
- 19 September 2021
- journal article
- research article
- Published by Hindawi Limited in International Journal of Intelligent Systems
- Vol. 37 (2) , 1222-1238
Abstract
Automatic image colorization without manual interventions is an ill-conditioned and inherently ambiguous problem. Most of existing methods focus on formulating colorization as a regression problem and learn parametric mappings from grayscale to color through deep neural networks. Due to the multimodalities of color-grayscale space, in many applications, it is not required to recover exact ground-truth color. Pair-wise pixel-to-pixel learning-based algorithms lack rationality. Techniques such as color space conversion techniques are then proposed to avoid such direct pixel learning. However, the coloring results after color space conversion are blunt and unnatural. In this paper, we hold viewpoints that a reasonable solution is to generate some colorized result that looks natural. No matter what color a region is to be assigned, the colorized region should be semantically and spatially consistent. In this paper, we propose an effective semantic-aware automatic colorization model via unpaired cycle-consistent self-supervised network. Low-level monochrome loss, perceptual identity loss and high-level semantic-consistence loss, together with adversarial loss, are introduced to guide network self-training. We train and test our model on randomly selected subsets from PASCAL VOC 2012. The experimental results including human subjective studies demonstrate that, compared with state-of-the-art methods, our proposed model can achieve more convincing and superior results. Relevant source code is available at https://github.com/YuSuen/ACCycleGAN.Keywords
This publication has 25 references indexed in Scilit:
- Visual attribute transfer through deep image analogyACM Transactions on Graphics, 2017
- Learning Representations for Automatic Colorization2016
- Let there be color!ACM Transactions on Graphics, 2016
- Deep Colorization2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Palette-based photo recoloringACM Transactions on Graphics, 2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Image colorization using similar images2012
- Colorization using optimizationACM Transactions on Graphics, 2004
- Transferring color to greyscale imagesACM Transactions on Graphics, 2002