FaceLit: Neural 3D Relightable Faces
Authors:
Preprint
Abstract
We propose a generative framework, FaceLit, capable of generating a 3D face that can be rendered at various user-defined lighting conditions and views, learned purely from 2D images in-the-wild without any manual annotation. Unlike existing works that require careful capture setup or human labor, we rely on off-the-shelf pose and illumination estimators. With these estimates, we incorporate the Phong reflectance model in the neural volume rendering framework. Our model learns to generate shape and material properties of a face such that, when rendered according to the natural statistics of pose and illumination, produces photorealistic face images with multiview 3D and illumination consistency. Our method enables photorealistic generation of faces with explicit illumination and view controls on multiple datasets - FFHQ, MetFaces and CelebA-HQ. We show state-of-the-art photorealism among 3D aware GANs on FFHQ dataset achieving an FID score of 3.5.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Generative Multiplane Images: Making a 2D GAN 3D-Aware2022
- Authentic volumetric avatars from a phone scanACM Transactions on Graphics, 2022
- Efficient Geometry-aware 3D Generative Adversarial Networks2022
- Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields2022
- MorphGAN: One-Shot Face Synthesis GAN for Detecting Recognition Bias2021
- A novel softplus linear unit for deep convolutional neural networksApplied Intelligence, 2017