Learning Pose-Aware Models for Pose-Invariant Face Recognition in the Wild
Authors:
- Iacopo MasiIMIacopo Masi
- Feng-Ju ChangFCFeng-Ju Chang
- Jongmoo ChoiJCJongmoo Choi
- Shai HarelSHShai Harel
- Jungyeon KimJKJungyeon Kim
- Kanggeon KimKKKanggeon Kim
- Jatuporn LeksutJLJatuporn Leksut
- Stephen RawlsSRStephen Rawls
- Yue WuYWYue Wu
- Tal HassnerTHTal Hassner
- Wael AbdAlmageedWAWael AbdAlmageed
- Gerard MedioniGMGerard Medioni
- 12 January 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 41 (2) , 379-393
Abstract
We propose a method designed to push the frontiers of unconstrained face recognition in the wild with an emphasis on extreme out-of-plane pose variations. Existing methods either expect a single model to learn pose invariance by training on massive amounts of data or else normalize images by aligning faces to a single frontal pose. Contrary to these, our method is designed to explicitly tackle pose variations. Our proposed Pose-Aware Models (PAM) process a face image using several pose-specific, deep convolutional neural networks (CNN). 3D rendering is used to synthesize multiple face poses from input images to both train these models and to provide additional robustness to pose variations at test time. Our paper presents an extensive analysis on the IJB-A benchmark, evaluating the effects that landmark detection accuracy, CNN layer selection, and pose model selection all have on the performance of the recognition pipeline. It further provides comparative evaluations on the IARPA Janus Benchmarks A (IJB-A) and the PIPA dataset. These tests show that our approach outperforms existing methods, even surprisingly matching the accuracy of methods that were specifically fine-tuned to the target dataset. Parts of this work previously appeared in [1] and [2].Keywords
Funding Information
- Office of the Director of National Intelligence
- Intelligence Advanced Research Projects Activity (IARPA 2014-14071600010)
This publication has 45 references indexed in Scilit:
- Deep Learning Face Representation from Predicting 10,000 Classes2014
- DeepFace: Closing the Gap to Human-Level Performance in Face Verification2014
- Viewing Real-World Faces in 3D2013
- Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild2013
- Image Classification with the Fisher Vector: Theory and PracticeInternational Journal of Computer Vision, 2013
- Using 3D Models to Recognize 2D Faces in the Wild2013
- Multi-PIEImage and Vision Computing, 2010
- Face recognition across pose: A reviewPattern Recognition, 2009
- Multi-subregion based probabilistic approach toward pose-invariant face recognition2004
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998