Preprint
Abstract
To enable realistic shape (e.g. pose and expression) transfer, existing face reenactment methods rely on a set of target faces for learning subject-specific traits. However, in real-world scenario end-users often only have one target face at hand, rendering existing methods inapplicable. In this work, we bridge this gap by proposing a novel one-shot face reenactment learning framework. Our key insight is that the one-shot learner should be able to disentangle and compose appearance and shape information for effective modeling. Specifically, the target face appearance and the source face shape are first projected into latent spaces with their corresponding encoders. Then these two latent spaces are associated by learning a shared decoder that aggregates multi-level features to produce the final reenactment results. To further improve the synthesizing quality on mustache and hair regions, we additionally propose FusionNet which combines the strengths of our learned decoder and the traditional warping method. Extensive experiments show that our one-shot face reenactment system achieves superior transfer fidelity as well as identity preserving capability than alternatives. More remarkably, our approach trained with only one target image per subject achieves competitive results to those using a set of target images, demonstrating the practical merit of this work. Code, models and an additional set of reenacted faces have been publicly released at the project page.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- ReenactGAN: Learning to Reenact Faces via Boundary Transfer2018
- X2Face: A Network for Controlling Face Generation Using Images, Audio, and Pose Codes2018
- Learning Face Age Progression: A Pyramid Architecture of GANs2018
- Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform2018
- PairedCycleGAN: Asymmetric Style Transfer for Applying and Removing Makeup2018
- Look at Boundary: A Boundary-Aware Face Alignment Algorithm2018
- Disentangled Person Image Generation2018
- Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild2017
- Disentangled Representation Learning GAN for Pose-Invariant Face Recognition2017
- Face2Face: Real-Time Face Capture and Reenactment of RGB Videos2016