TraVeLGAN: Image-To-Image Translation by Transformation Vector Learning
Authors:
- Smita KrishnaswamySKSmita Krishnaswamy
- 1 June 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8975-8984
Abstract
Interest in image-to-image translation has grown substantially in recent years with the success of unsupervised models based on the cycle-consistency assumption. The achievements of these models have been limited to a particular subset of domains where this assumption yields good results, namely homogeneous domains that are characterized by style or texture differences. We tackle the challenging problem of image-to-image translation where the domains are defined by high-level shapes and contexts, as well as including significant clutter and heterogeneity. For this purpose, we introduce a novel GAN based on preserving intra-domain vector transformations in a latent space learned by a siamese network. The traditional GAN system introduced a discriminator network to guide the generator into generating images in the target domain. To this two-network system we add a third: a siamese network that guides the generator so that each original image shares semantics with its generated version. With this new three-network system, we no longer need to constrain the generators with the ubiquitous cycle-consistency restraint or any other autoencoding regularization. As a result, the generators can learn mappings between more complex domains that differ from each other by more than just style or texture. We demonstrate our model by mapping between high-resolution, arbitrarily chosen classes from the Imagenet dataset completely without pre-processing such as cropping, centering, or filtering unrepresentative images.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- DualGAN: Unsupervised Dual Learning for Image-to-Image Translation2017
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks2017
- Image-to-Image Translation with Conditional Adversarial Networks2017
- Neural Information Processing2017
- Image Style Transfer Using Convolutional Neural Networks2016
- Siamese network features for image matching2016
- Semi-supervised Domain Adaptation with Subspace Learning for visual recognition2015
- Going deeper with convolutions2015
- Style Transfer Via Image Component AnalysisIEEE Transactions on Multimedia, 2013