FSGAN: Subject Agnostic Face Swapping and Reenactment
Authors:
Top Cited Papers
- 1 October 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7183-7192
Abstract
We present Face Swapping GAN (FSGAN) for face swapping and reenactment. Unlike previous work, FSGAN is subject agnostic and can be applied to pairs of faces without requiring training on those faces. To this end, we describe a number of technical contributions. We derive a novel recurrent neural network (RNN)-based approach for face reenactment which adjusts for both pose and expression variations and can be applied to a single image or a video sequence. For video sequences, we introduce continuous interpolation of the face views based on reenactment, Delaunay Triangulation, and barycentric coordinates. Occluded face regions are handled by a face completion network. Finally, we use a face blending network for seamless blending of the two faces while preserving target skin color and lighting conditions. This network uses a novel Poisson blending loss which combines Poisson optimization with perceptual loss. We compare our approach to existing state-of-the-art systems and show our results to be both qualitatively and quantitatively superior.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- Figaro, hair detection and segmentation in the wild2016
- Demo of Face2Face2016
- Transfiguring portraitsACM Transactions on Graphics, 2016
- Effective face frontalization in unconstrained images2015
- Augmenting CRFs with Boltzmann Machine Shape Priors for Image Labeling2013
- Face swappingACM Transactions on Graphics, 2008
- Exchanging Faces in ImagesComputer Graphics Forum, 2004
- Face recognition based on fitting a 3D morphable modelIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003
- Poisson image editingACM Transactions on Graphics, 2003
- Face identification across different poses and illuminations with a 3D morphable model2003