Lightweight Photometric Stereo for Facial Details Recovery
Preprint
Abstract
Recently, 3D face reconstruction from a single image has achieved great success with the help of deep learning and shape prior knowledge, but they often fail to produce accurate geometry details. On the other hand, photometric stereo methods can recover reliable geometry details, but require dense inputs and need to solve a complex optimization problem. In this paper, we present a lightweight strategy that only requires sparse inputs or even a single image to recover high-fidelity face shapes with images captured under near-field lights. To this end, we construct a dataset containing 84 different subjects with 29 expressions under 3 different lights. Data augmentation is applied to enrich the data in terms of diversity in identity, lighting, expression, etc. With this constructed dataset, we propose a novel neural network specially designed for photometric stereo based 3D face reconstruction. Extensive experiments and comparisons demonstrate that our method can generate high-quality reconstruction results with one to three facial images captured under near-field lights. Our full framework is available at https://github.com/Juyong/FacePSNet.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- 3D Face Reconstruction With Geometry Details From a Single ImageIEEE Transactions on Image Processing, 2018
- CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face ImagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- LED-Based Photometric Stereo: Modeling, Calibration and Numerical SolutionJournal of Mathematical Imaging and Vision, 2017
- FaceWarehouse: A 3D Facial Expression Database for Visual ComputingIEEE Transactions on Visualization and Computer Graphics, 2013
- An introduction to image-based 3D surface reconstruction and a survey of photometric stereo methods3D Research, 2011
- Effect of illumination on automatic expression recognition: A novel 3D relightable facial database2011
- The Digital Emily Project: Achieving a Photorealistic Digital ActorIEEE Computer Graphics and Applications, 2010
- IMAGE BASED 3D FACE RECONSTRUCTION: A SURVEYInternational Journal of Image and Graphics, 2009
- Face recognition based on fitting a 3D morphable modelIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003