Action Unit Driven Facial Expression Synthesis from a Single Image with Patch Attentive GAN
- 17 March 2021
- journal article
- research article
- Published by Wiley in Computer Graphics Forum
- Vol. 40 (6) , 47-61
Abstract
Recent advances in generative adversarial networks (GANs) have shown tremendous success for facial expression generation tasks. However, generating vivid and expressive facial expressions at Action Units (AUs) level is still challenging, due to the fact that automatic facial expression analysis for AU intensity itself is an unsolved difficult task. In this paper, we propose a novel synthesis‐by‐analysis approach by leveraging the power of GAN framework and state‐of‐the‐art AU detection model to achieve better results for AU‐driven facial expression generation. Specifically, we design a novel discriminator architecture by modifying the patch‐attentive AU detection network for AU intensity estimation and combine it with a global image encoder for adversarial learning to force the generator to produce more expressive and realistic facial images. We also introduce a balanced sampling approach to alleviate the imbalanced learning problem for AU synthesis. Extensive experimental results on DISFA and DISFA+ show that our approach outperforms the state‐of‐the‐art in terms of photo‐realism and expressiveness of the facial expression quantitatively and qualitatively.Keywords
Funding Information
- China Scholarship Council (201506290085)
This publication has 36 references indexed in Scilit:
- Real-time facial animation with image-based dynamic avatarsACM Transactions on Graphics, 2016
- Reconstruction of Personalized 3D Face Rigs from Monocular VideoACM Transactions on Graphics, 2016
- Estimating smile intensity: A better wayPattern Recognition Letters, 2015
- Cross-dataset learning and person-specific normalisation for automatic Action Unit detection2015
- Deep learning based FACS Action Unit occurrence and intensity estimation2015
- FaceWarehouse: A 3D Facial Expression Database for Visual ComputingIEEE Transactions on Visualization and Computer Graphics, 2013
- Regression-based intensity estimation of facial action unitsImage and Vision Computing, 2012
- Face transfer with multilinear models2005
- Reanimating Faces in Images and VideoComputer Graphics Forum, 2003
- A morphable model for the synthesis of 3D faces1999