Appearance and Pose-Conditioned Human Image Generation Using Deformable GANs
- 16 October 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 43 (4) , 1156-1171
Abstract
In this paper, we address the problem of generating person images conditioned on both pose and appearance information. Specifically, given an image $x_a$ of a person and a target pose $P(x_b)$ , extracted from an image $x_b$ , we synthesize a new image of that person in pose $P(x_b)$ , while preserving the visual details in $x_a$ . In order to deal with pixel-to-pixel misalignments caused by the pose differences between $P(x_a)$ and $P(x_b)$ , we introduce deformable skip connections in the generator of our Generative Adversarial Network. Moreover, a nearest-neighbour loss is proposed instead of the common $L_1$ and $L_2$ losses in order to match the details of the generated image with the target image. Quantitative and qualitative results, using common datasets and protocols recently proposed for this task, show that our approach is competitive with respect to the state of the art. Moreover, we conduct an extensive evaluation using off-the-shell person re-identification (Re-ID) systems trained with person-generation based augmented data, which is one of the main important applications for this task. Our experiments show that our Deformable GANs can significantly boost the Re-ID accuracy and are even better than data-augmentation methods specifically trained using Re-ID losses.
Keywords
All Related Versions
Funding Information
- European Research Council (788793-BACKUP)
This publication has 33 references indexed in Scilit:
- Visual attribute transfer through deep image analogyACM Transactions on Graphics, 2017
- SSD: Single Shot MultiBox Detector2016
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution2016
- Context Encoders: Feature Learning by Inpainting2016
- Rethinking the Inception Architecture for Computer Vision2016
- DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations2016
- Scalable Person Re-identification: A Benchmark2015
- Person re-identification by Local Maximal Occurrence representation and metric learning2015
- Large scale metric learning from equivalence constraints2012
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004