Local Spherical Harmonics for Facial Shape and Albedo Estimation
Open Access
- 24 September 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 177424-177436
Abstract
In this paper, we present a novel facial albedo and 3D shape recovery method with a local spherical harmonic illumination model. From a face in an image, the proposed method can produce a high-quality 3D shape and albedo using a novel parameterization of local illuminations. Because a facial shape is partially convex, a single spherical harmonics is generally used for the illumination of a face within a constrained illumination environment. However, when a facial image is captured in an unconstrained scene, the illumination is inappropriately estimated due to the presence of shadow and specular reflections. To address this issue, we propose a novel local spherical harmonic illumination model for representing the illumination of a face. Unlike the existing parameterization of local illumination, our local spherical harmonic illumination model utilizes a smooth weight function for the seamless representation of natural illumination. Therefore, the albedo and shape information in an image can be precisely estimated using the first-order spherical harmonics. For accurate estimation of albedo, we also utilize facial albedo statistics to prevent the estimated albedo from becoming biased toward input image. Furthermore, we developed an accurate and reliable 3D shape reconstruction method from a normal map based on tetrahedron-based deformation. Comparing to the Laplacian deformation based method, our method is applicable to any mesh regardless of its structure. Through rigorous experiments, we demonstrate that the proposed local spherical harmonic illumination model is effective in estimating the complex illumination and can recover a high-quality facial albedo and 3D shape.Keywords
Funding Information
- Ministry of Culture, Sports and Tourism (MCST) and from Korea Copyright Commission in 2020
This publication has 46 references indexed in Scilit:
- Acquiring reflectance and shape from continuous spherical harmonic illuminationACM Transactions on Graphics, 2013
- Multiview face capture using polarized spherical gradient illuminationACM Transactions on Graphics, 2011
- Analysis of human faces using a measurement-based skin reflectance modelACM Transactions on Graphics, 2006
- Deformation transfer for triangle meshesACM Transactions on Graphics, 2004
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Accuracy of Spherical Harmonic Approximations for Images of Lambertian Objects under Far and Near Lighting2004
- Face recognition based on fitting a 3D morphable modelIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003
- Lambertian reflectance and linear subspacesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003
- From few to many: illumination cone models for face recognition under variable lighting and poseIEEE Transactions on Pattern Analysis and Machine Intelligence, 2001
- Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear SystemsSIAM Journal on Scientific and Statistical Computing, 1992