Learning 3-D Face Shape From Diverse Sources With Cross-Domain Face Synthesis
- 29 July 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE MultiMedia
- Vol. 30 (1) , 7-16
Abstract
Monocular face reconstruction is a significant task in many multimedia applications. However, learning-based methods unequivocally suffer from the lack of large datasets annotated with 3D ground-truth. To tackle this problem, we proposed a novel end-to-end 3D face reconstruction network consisting of a domain-transfer conditional GAN (cGAN) and a face reconstruction network. Our method first uses cGAN to translate the realistic face images to the specific rendered style, with a novel 2D facial edge consistency loss function to exploit in-the-wild images. The domain-transferred images are then fed into 3D face reconstruction network. We further propose a novel reprojection consistency loss to restrict 3D face reconstruction network in a self-supervised way. Our approach can be trained with annotated dataset, synthetic dataset and in-the-wild images to learn a unified face model. Extensive experiments have demonstrated the effectiveness of our method.Keywords
Funding Information
- National Key Research and Development Plan of China (2020YFC2006000)
- National Natural Science Foundation of China (72071089)
This publication has 18 references indexed in Scilit:
- Semi-Supervised Monocular 3D Face Reconstruction With End-to-End Shape-Preserved Domain Transfer2019
- Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network2018
- Face Alignment in Full Pose Range: A 3D Total SolutionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks2017
- Image-to-Image Translation with Conditional Adversarial Networks2017
- 3D Face Reconstruction by Learning from Synthetic Data2016
- Displaced dynamic expression regression for real-time facial tracking and animationACM Transactions on Graphics, 2014
- Single view-based 3D face reconstruction robust to self-occlusionEURASIP Journal on Advances in Signal Processing, 2012
- Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization2011
- A morphable model for the synthesis of 3D faces1999