Multi-view 3D face reconstruction with deep recurrent neural networks
- 1 October 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 483-492
Abstract
Image-based 3D face reconstruction has great potential in different areas, such as facial recognition, facial analysis, and facial animation. Due to the variations in image quality, single-image-based 3D face reconstruction might not be sufficient to accurately reconstruct a 3D face. To overcome this limitation, multi-view 3D face reconstruction uses multiple images of the same subject and aggregates complementary information for better accuracy. Though theoretically appealing, there are multiple challenges in practice. Among these challenges, the most significant is that it is difficult to establish coherent and accurate correspondence among a set of images, especially when these images are captured in different conditions. In this paper, we propose a method, Deep Recurrent 3D FAce Reconstruction (DRFAR), to solve the task ofmulti-view 3D face reconstruction using a subspace representation of the 3D facial shape and a deep recurrent neural network that consists of both a deep con-volutional neural network (DCNN) and a recurrent neural network (RNN). The DCNN disentangles the facial identity and the facial expression components for each single image independently, while the RNN fuses identity-related features from the DCNN and aggregates the identity specific contextual information, or the identity signal, from the whole set of images to predict the facial identity parameter, which is robust to variations in image quality and is consistent over the whole set of images. Through extensive experiments, we evaluate our proposed method and demonstrate its superiority over existing methods.Keywords
This publication has 25 references indexed in Scilit:
- Face Alignment Across Large Poses: A 3D Solution2016
- Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting2016
- Rendering or normalization? An analysis of the 3D-aided pose-invariant face recognition2016
- Internet Based Morphable Model2013
- A SFM-based sparse to dense 3D face reconstruction method robust to feature tracking errors2013
- Towards Linear-Time Incremental Structure from Motion2013
- Face reconstruction in the wild2011
- A SfM-based 3D face reconstruction method robust to self-occlusion by using a shape conversion matrixPattern Recognition, 2011
- Accurate 3D face reconstruction from weakly calibrated wide baseline images with profile contours2010
- FRVT 2006 and ICE 2006 Large-Scale Experimental ResultsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009