Real-Time 3d Face Reconstruction From Single Image Using End-To-End Cnn Regression
Authors:
- 19 September 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3293-3297
Abstract
This paper presents a learning-based method for detailed 3D face reconstruction from a single unconstrained image. The core of our method is an end-to-end multi-task network architecture. The purpose of the proposed network is to predict a geometric representation of 3D face from a given facial image. Unlike most existing reconstruction methods using low-dimension morphable models, we propose a pixel-based multi-scale representation of a detailed 3D face to ensure that our reconstruction results are not limited by the expressiveness of linear models. We break the task of high-fidelity face reconstruction into three subtasks, which are face region segmentation, coarse-scale reconstruction and detail recovery. So the end-to-end network is constructed as a multitask mode, which contains three subtask networks to deal with different subtasks respectively. A backbone network with feature pyramid structure is proposed as well to provide different levels of feature maps required by the three subtask networks. We train our end-to-end network in the spirit of the recent photo-realistic data generation approach. The experimental results demonstrate that our method can work with totally unconstrained images and produce high-quality reconstruction but with less runtime compared to the state-of-the-art.Keywords
Funding Information
- National Natural Science Foundation of China
This publication has 19 references indexed in Scilit:
- Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network2017
- Pyramid Scene Parsing Network2017
- 3D Face Morphable Models "In-the-Wild"2017
- Image-to-Image Translation with Conditional Adversarial Networks2017
- 3D Face Reconstruction by Learning from Synthetic Data2016
- Deep Residual Learning for Image Recognition2016
- FlowNet: Learning Optical Flow with Convolutional Networks2015
- 300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge2013
- Multiview face capture using polarized spherical gradient illuminationACM Transactions on Graphics, 2011
- A morphable model for the synthesis of 3D faces1999