Geometry-Aware GAN for Face Attribute Transfer
Open Access
- 18 September 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 145953-145969
Abstract
In this paper, the geometry-aware GAN, referred to as GAGAN, is proposed to address the issue of face attribute transfer with unpaired data. The source and target images are not aligned and come from different individuals. The key idea is to deform the source image according to the geometry features and generate a high-resolution image with the desired attribute. To address the problem of the unpaired training samples, the CycleGAN architecture is applied to form an attribute adding and removing cycle, where the bilateral mappings between the source and target domains are learned. The geometry flow and occlusion mask are learned by the warping sub-network to capture the geometric variation between the two domains. In the attribute adding process, the spatial transformer network (STN) warps the source face into the desired pose and shape according to the flow, and the transfer sub-network hallucinates new components on the warped image. In the attribute removing process, the recover sub-network and the STN reverts the sample back to the source domain. Experiments on the benchmarks CELEBA and CELEBA-HQ datasets demonstrate the advantages of our method compared to the baselines, in terms of both quantitative and qualitative evaluation.Keywords
Funding Information
- National Natural Science Foundation of China (61622110, 61629101, 2018YFB1800804)
This publication has 31 references indexed in Scilit:
- Beyond Trace Ratio: Weighted Harmonic Mean of Trace Ratios for Multiclass Discriminant AnalysisIEEE Transactions on Knowledge and Data Engineering, 2017
- Feature Interaction Augmented Sparse Learning for Fast Kinect Motion DetectionIEEE Transactions on Image Processing, 2017
- Attribute2Image: Conditional Image Generation from Visual Attributes2016
- Semantic Pooling for Complex Event Analysis in Untrimmed VideosIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Demo of Face2Face2016
- Real-time facial animation with image-based dynamic avatarsACM Transactions on Graphics, 2016
- Adaptive 3D Face Reconstruction from Unconstrained Photo Collections2016
- Deep Learning Face Attributes in the Wild2015
- Expression flow for 3D-aware face component transferACM Transactions on Graphics, 2011
- Visio-lizationACM Transactions on Graphics, 2009