Photographic Text-to-Image Synthesis with a Hierarchically-Nested Adversarial Network
Authors:
- Zizhao ZhangZZZizhao Zhang
- Yuanpu XieYXYuanpu Xie
- Lin YangLYLin Yang
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6199-6208
Abstract
This paper presents a novel method to deal with the challenging task of generating photographic images conditioned on semantic image descriptions. Our method introduces accompanying hierarchical-nested adversarial objectives inside the network hierarchies, which regularize mid-level representations and assist generator training to capture the complex image statistics. We present an extensile single-stream generator architecture to better adapt the jointed discriminators and push generated images up to high resolutions. We adopt a multi-purpose adversarial loss to encourage more effective image and text information usage in order to improve the semantic consistency and image fidelity simultaneously. Furthermore, we introduce a new visual-semantic similarity measure to evaluate the semantic consistency of generated images. With extensive experimental validation on three public datasets, our method significantly improves previous state of the arts on all datasets over different evaluation metrics.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Photographic Image Synthesis with Cascaded Refinement Networks2017
- Adversarial nets with perceptual losses for text-to-image synthesis2017
- Learning from Simulated and Unsupervised Images through Adversarial Training2017
- Stacked Generative Adversarial Networks2017
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
- Pyramid Scene Parsing Network2017
- Image-to-Image Translation with Conditional Adversarial Networks2017
- Holistically-Nested Edge Detection2015
- Going deeper with convolutions2015
- Fully convolutional networks for semantic segmentation2015