LT-GAN: Self-Supervised GAN with Latent Transformation Detection
Preprint
Abstract
Generative Adversarial Networks (GANs) coupled with self-supervised tasks have shown promising results in unconditional and semi-supervised image generation. We propose a self-supervised approach (LT-GAN) to improve the generation quality and diversity of images by estimating the GAN-induced transformation (i.e. transformation induced in the generated images by perturbing the latent space of generator). Specifically, given two pairs of images where each pair comprises of a generated image and its transformed version, the self-supervision task aims to identify whether the latent transformation applied in the given pair is same to that of the other pair. Hence, this auxiliary loss encourages the generator to produce images that are distinguishable by the auxiliary network, which in turn promotes the synthesis of semantically consistent images with respect to latent transformations. We show the efficacy of this pretext task by improving the image generation quality in terms of FID on state-of-the-art models for both conditional and unconditional settings on CIFAR-10, CelebA-HQ and ImageNet datasets. Moreover, we empirically show that LT-GAN helps in improving controlled image editing for CelebA-HQ and ImageNet over baseline models. We experimentally demonstrate that our proposed LT self-supervision task can be effectively combined with other state-of-the-art training techniques for added benefits. Consequently, we show that our approach achieves the new state-of-the-art FID score of 9.8 on conditional CIFAR-10 image generation.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Deep Clustering for Unsupervised Learning of Visual Features2018
- Audio-Visual Scene Analysis with Self-Supervised Multisensory Features2018
- Boosting Self-Supervised Learning via Knowledge Transfer2018
- Learning and Using the Arrow of Time2018
- Improvements to Context Based Self-Supervised Learning2018
- Deep Feature Interpolation for Image Content Changes2017
- Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction2017
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles2016
- Context Encoders: Feature Learning by Inpainting2016
- Deep Residual Learning for Image Recognition2016