Three-Dimensional Face Reconstruction Using Multi-View-Based Bilinear Model
Open Access
Abstract
Face reconstruction is a popular topic in 3D vision system. However, traditional methods often depend on monocular cues, which contain few feature pixels and only use their location information while ignoring a lot of textural information. Furthermore, they are affected by the accuracy of the feature extraction method and occlusion. Here, we propose a novel facial reconstruction framework that accurately extracts the 3D shapes and poses of faces from images captured at multi-views. It extends the traditional method using the monocular bilinear model to the multi-view-based bilinear model by incorporating the feature prior constraint and the texture constraint, which are learned from multi-view images. The feature prior constraint is used as a shape prior to allowing us to estimate accurate 3D facial contours. Furthermore, the texture constraint extracts a high-precision 3D facial shape where traditional methods fail because of their limited number of feature points or the mostly texture-less and texture-repetitive nature of the input images. Meanwhile, it fully explores the implied 3D information of the multi-view images, which also enhances the robustness of the results. Additionally, the proposed method uses only two or more uncalibrated images with an arbitrary baseline, estimating calibration and shape simultaneously. A comparison with the state-of-the-art monocular bilinear model-based method shows that the proposed method has a significantly higher level of accuracy.Keywords
Funding Information
- Natural Science Foundation of Hebei Province (F2017205066)
- National Natural Science Foundation of China (61802109)
- Science Foundation of Hebei Normal University (L2017B06, L2018K02)
This publication has 46 references indexed in Scilit:
- A Multi-Modal Face Recognition Method Using Complete Local Derivative Patterns and Depth MapsSensors, 2014
- A Two-Stage Framework for 3D FaceReconstruction from RGBD ImagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Parallel L-BFGS-B algorithm on GPUComputers & Graphics, 2014
- Random-Profiles-Based 3D Face Recognition SystemSensors, 2014
- An efficient 3D face recognition approach using local geometrical signaturesPattern Recognition, 2014
- Fast 3D face reconstruction based on uncalibrated photometric stereoMultimedia Tools and Applications, 2013
- FaceWarehouse: A 3D Facial Expression Database for Visual ComputingIEEE Transactions on Visualization and Computer Graphics, 2013
- 3D Reconstruction from Multiple Images Part 1: PrinciplesFoundations and Trends in Computer Graphics and Vision, 2010
- Face transfer with multilinear modelsACM Transactions on Graphics, 2005
- Face recognition based on fitting a 3D morphable modelIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003