Pose-variant 3D Facial Attribute Generation
Preprint
Abstract
We address the challenging problem of generating facial attributes using a single image in an unconstrained pose. In contrast to prior works that largely consider generation on 2D near-frontal images, we propose a GAN-based framework to generate attributes directly on a dense 3D representation given by UV texture and position maps, resulting in photorealistic, geometrically-consistent and identity-preserving outputs. Starting from a self-occluded UV texture map obtained by applying an off-the-shelf 3D reconstruction method, we propose two novel components. First, a texture completion generative adversarial network (TC-GAN) completes the partial UV texture map. Second, a 3D attribute generation GAN (3DA-GAN) synthesizes the target attribute while obtaining an appearance consistent with 3D face geometry and preserving identity. Extensive experiments on CelebA, LFW and IJB-A show that our method achieves consistently better attribute generation accuracy than prior methods, a higher degree of qualitative photorealism and preserves face identity information.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- GANimation: Anatomically-Aware Facial Animation from a Single Image2018
- Learning Residual Images for Face Attribute Manipulation2017
- Disentangled Representation Learning GAN for Pose-Invariant Face Recognition2017
- Deep Feature Interpolation for Image Content Changes2017
- DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild2017
- Attribute2Image: Conditional Image Generation from Visual Attributes2016
- Face Alignment Across Large Poses: A 3D Solution2016
- High-fidelity Pose and Expression Normalization for face recognition in the wild2015
- Effective face frontalization in unconstrained images2015
- Rotating your face using multi-task deep neural network2015