EGGAN: Learning Latent Space for Fine-Grained Expression Manipulation
- 25 February 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE MultiMedia
- Vol. 28 (3) , 42-51
Abstract
Fine-grained facial expression aims at changing the expression of an image without altering facial identity. Most current expression manipulation methods are based on a discrete expression label, which mainly manipulates holistic expression with details neglected. To handle the above mentioned problems, we propose an end-to-end expression-guided generative adversarial network (EGGAN), which synthesizes an image with expected expression given continuous expression label and structured latent code. In particular, an adversarial autoencoder is used to translate a source image into a structured latent space. The encoded latent code and the target expression label are input to a conditional GAN to synthesize an image with the target expression. Moreover, a perceptual loss and a multiscale structural similarity loss are introduced to preserve facial identity and global shape during expression manipulation. Extensive experiments demonstrate that our approach can edit fine-grained expressions, and synthesize continuous intermediate expressions between source and target expressions.Keywords
Funding Information
- National Key Research and Development Program of China (2019YFC1521104)
- National Natural Science Foundation of China (61972157)
This publication has 13 references indexed in Scilit:
- Controllable Image-to-Video Translation: A Case Study on Facial Expression GenerationProceedings of the AAAI Conference on Artificial Intelligence, 2019
- AttGAN: Facial Attribute Editing by Only Changing What You WantIEEE Transactions on Image Processing, 2019
- GANimation: Anatomically-Aware Facial Animation from a Single Image2018
- StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation2018
- A Light CNN for Deep Face Representation With Noisy LabelsIEEE Transactions on Information Forensics and Security, 2018
- OpenFace 2.0: Facial Behavior Analysis Toolkit2018
- Loss Functions for Image Restoration With Neural NetworksIEEE Transactions on Computational Imaging, 2016
- Perceptual Losses for Real-Time Style Transfer and Super-Resolution2016
- Rethinking the Inception Architecture for Computer Vision2016
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004