TransGaGa: Geometry-Aware Unsupervised Image-to-Image Translation
Preprint
Abstract
Unsupervised image-to-image translation aims at learning a mapping between two visual domains. However, learning a translation across large geometry variations always ends up with failure. In this work, we present a novel disentangle-and-translate framework to tackle the complex objects image-to-image translation task. Instead of learning the mapping on the image space directly, we disentangle image space into a Cartesian product of the appearance and the geometry latent spaces. Specifically, we first introduce a geometry prior loss and a conditional VAE loss to encourage the network to learn independent but complementary representations. The translation is then built on appearance and geometry space separately. Extensive experiments demonstrate the superior performance of our method to other state-of-the-art approaches, especially in the challenging near-rigid and non-rigid objects translation tasks. In addition, by taking different exemplars as the appearance references, our method also supports multimodal translation. Project page: https://wywu.github.io/projects/TGaGa/TGaGa.htmlKeywords
All Related Versions
This publication has 24 references indexed in Scilit:
- DRIT++: Diverse Image-to-Image Translation via Disentangled RepresentationsInternational Journal of Computer Vision, 2020
- ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks2019
- Multimodal Unsupervised Image-to-Image Translation2018
- ReenactGAN: Learning to Reenact Faces via Boundary Transfer2018
- Multi-context Attention for Human Pose Estimation2017
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
- Image-to-Image Translation with Conditional Adversarial Networks2017
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution2016
- Deep Residual Learning for Image Recognition2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015