Unsupervised Training for 3D Morphable Model Regression
Authors:
- Kyle GenovaKGKyle Genova
- Forrester ColeFCForrester Cole
- Aaron MaschinotAMAaron Maschinot
- Aaron SarnaASAaron Sarna
- Daniel VlasicDVDaniel Vlasic
- William T. FreemanWFWilliam T. Freeman
- 1 June 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8377-8386
Abstract
We present a method for training a regression network from image pixels to 3D morphable model coordinates using only unlabeled photographs. The training loss is based on features from a facial recognition network, computed on-the-fly by rendering the predicted faces with a differentiable renderer. To make training from features feasible and avoid network fooling effects, we introduce three objectives: a batch distribution loss that encourages the output distribution to match the distribution of the morphable model, a loopback loss that ensures the network can correctly reinterpret its own output, and a multi-view identity loss that compares the features of the predicted 3D face and the input photograph from multiple viewing angles. We train a regression network using these objectives, a set of unlabeled photographs, and the morphable model itself, and demonstrate state-of-the-art results.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- 3D Face Reconstruction by Learning from Synthetic Data2016
- The MegaFace Benchmark: 1 Million Faces for Recognition at Scale2016
- FaceNet: A unified embedding for face recognition and clustering2015
- DeepFace: Closing the Gap to Human-Level Performance in Face Verification2014
- The florence 2D/3D hybrid face dataset2011
- State-of-the-art of 3D facial reconstruction methods for face recognition based on a single 2D training image per personPattern Recognition Letters, 2009
- Face recognition based on fitting a 3D morphable modelIEEE Transactions on Pattern Analysis and Machine Intelligence, 2003
- Reanimating Faces in Images and VideoComputer Graphics Forum, 2003
- A morphable model for the synthesis of 3D faces1999
- Illumination for computer generated picturesCommunications of the ACM, 1975