Semantic Photo Manipulation with a Generative Image Prior
Preprint
Abstract
Despite the recent success of GANs in synthesizing images conditioned on inputs such as a user sketch, text, or semantic labels, manipulating the high-level attributes of an existing natural photograph with GANs is challenging for two reasons. First, it is hard for GANs to precisely reproduce an input image. Second, after manipulation, the newly synthesized pixels often do not fit the original image. In this paper, we address these issues by adapting the image prior learned by GANs to image statistics of an individual image. Our method can accurately reconstruct the input image and synthesize new content, consistent with the appearance of the input image. We demonstrate our interactive system on several semantic image editing tasks, including synthesizing new objects consistent with background, removing unwanted objects, and changing the appearance of an object. Quantitative and qualitative comparisons against several existing methods demonstrate the effectiveness of our method.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Semantic Image Synthesis With Spatially-Adaptive Normalization2019
- A Closed-Form Solution to Photorealistic Image Stylization2018
- Zero-Shot Super-Resolution Using Deep Internal Learning2018
- StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks2017
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Scribbler: Controlling Deep Image Synthesis with Sketch and Color2017
- Image Style Transfer Using Convolutional Neural Networks2016
- Generative Visual Manipulation on the Natural Image Manifold2016
- Context Encoders: Feature Learning by Inpainting2016
- Color transfer between imagesIEEE Computer Graphics and Applications, 2001