Face Photo-Sketch Recognition Using Bidirectional Collaborative Synthesis Network
Authors:
- 3 January 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
This research features a deep learning based framework to address the problem of matching a given face sketch image against a face photo database. The photo-sketch matching problem is challenging because 1) modality gap between photo and sketch is very large, and 2) the number of paired photo/ sketch data is insufficient to train deep network. To circumvent the problem of large modality gap, our approach is to use an intermediate latent space between the two modalities. We effectively align the distributions of the two modalities in this latent space by employing a bidirectional (photo → sketch and sketch → photo) collaborative synthesis network. A StyleGAN-like architecture is utilized to make the intermediate latent space be equipped with rich representation power. To resolve the problem of insufficient training samples, we introduce a three-step training scheme. Extensive evaluation on public composite face sketch database confirms superior performance of our method compared to existing state-of-the-art methods. The proposed methodology can be employed in matching other modality pairs.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Graphical Representation for Heterogeneous Face RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Deep Learning Face Attributes in the Wild2015
- Recognizing composite sketches with digital face images via SSD dictionary2014
- Data insufficiency in sketch versus photo face recognition2012
- Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch2011
- Coupled information-theoretic encoding for face photo-sketch recognition2011
- Effective Unconstrained Face Recognition by Combining Multiple Descriptors and Learned Background StatisticsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- A Nonlinear Approach for Face Sketch Synthesis and Recognition2005
- Face sketch synthesis and recognition2003
- The FERET evaluation methodology for face-recognition algorithmsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000