A Deep Collaborative Framework for Face Photo–Sketch Synthesis
Authors:
- 22 January 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 30 (10) , 3096-3108
Abstract
Great breakthroughs have been made in the accuracy and speed of face photo-sketch synthesis in recent years. Regression-based methods have gained increasing attention, which benefit from deeper and faster end-to-end convolutional neural networks. However, most of these models typically formulate the mapping from photo domain X to sketch domain Y as a unidirectional feedforward mapping, G: X → Y , and vice versa, F: Y → X ; thus, the utilization of mutual interaction between two opposite mappings is lacking. Therefore, we proposed a collaborative framework for face photo-sketch synthesis. The concept behind our model was that a middle latent domain ~Z between the photo domain X and the sketch domain Y can be learned during the learning procedure of G: X → Y and F: Y → X by introducing a collaborative loss that makes full use of two opposite mappings. This strategy can constrain the two opposite mappings and make them more symmetrical, thus making the network more suitable for the photo-sketch synthesis task and obtaining higher quality generated images. Qualitative and quantitative experiments demonstrated the superior performance of our model in comparison with the existing state-of-the-art solutions.Keywords
Funding Information
- National Natural Science Foundation of China (61671339, 61876142, 61432014, U1605252, 61772402, 61501339)
- National Key Research and Development Program of China (2016QY01W0200)
- Key Industrial Innovation Chain in Industrial Domain (2016KTZDGY04-02)
- National High-Level Talents Special Support Program of China (CS31117200001)
- Young Elite Scientists Sponsorship Program by CAST (2016QNRC001)
- Natural Science Basic Research Plan in the Shaanxi Province of China (2017JM6085)
- Young Talent Fund of the University Association for Science and Technology in Shaanxi, China
- Fundamental Research Funds for the Central Universities (XJS17086)
- Xidian University
- CCF-Tencent Open Fund
- Xidian University—Intellifusion Joint Innovation Laboratory of Artificial Intelligence
This publication has 40 references indexed in Scilit:
- Multiple Representations-Based Face Sketch–Photo SynthesisIEEE Transactions on Neural Networks and Learning Systems, 2015
- End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning2015
- Face Sketch Synthesis via Sparse Representation-Based Greedy SearchIEEE Transactions on Image Processing, 2015
- A Comprehensive Survey to Face HallucinationInternational Journal of Computer Vision, 2013
- Robust Kernel Representation With Statistical Local Features for Face RecognitionIEEE Transactions on Neural Networks and Learning Systems, 2013
- Heterogeneous image transformationPattern Recognition Letters, 2013
- Face Sketch Synthesis via Multivariate Output Regression2011
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- A new LDA-based face recognition system which can solve the small sample size problemPattern Recognition, 2000
- The FERET evaluation methodology for face-recognition algorithmsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000