Implicit Neural Deformation for Sparse‐View Face Reconstruction
- 1 October 2022
- journal article
- conference paper
- Published by Wiley in Computer Graphics Forum
- Vol. 41 (7) , 601-610
Abstract
In this work, we present a new method for 3D face reconstruction from sparse‐view RGB images. Unlike previous methods which are built upon 3D morphable models (3DMMs) with limited details, we leverage an implicit representation to encode rich geometric features. Our overall pipeline consists of two major components, including a geometry network, which learns a deformable neural signed distance function (SDF) as the 3D face representation, and a rendering network, which learns to render on‐surface points of the neural SDF to match the input images via self‐supervised optimization. To handle in‐the‐wild sparse‐view input of the same target with different expressions at test time, we propose residual latent code to effectively expand the shape space of the learned implicit face representation as well as a novel view‐switch loss to enforce consistency among different views. Our experimental results on several benchmark datasets demonstrate that our approach outperforms alternative baselines and achieves superior face reconstruction results compared to state‐of‐the‐art methods.Keywords
All Related Versions
This publication has 49 references indexed in Scilit:
- Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network2017
- Reconstruction of Personalized 3D Face Rigs from Monocular VideoACM Transactions on Graphics, 2016
- Real-Time Face Reconstruction from a Single Depth Image2014
- High resolution passive facial performance captureACM Transactions on Graphics, 2010
- High-quality single-shot capture of facial geometryACM Transactions on Graphics, 2010
- Bosphorus Database for 3D Face Analysis2008
- Face recognition based on fitting a 3D morphable modelIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003
- A morphable model for the synthesis of 3D faces1999
- Marching cubes: A high resolution 3D surface construction algorithmACM SIGGRAPH Computer Graphics, 1987
- A Generalized Solution of the Orthogonal Procrustes ProblemPsychometrika, 1966