A Neural Virtual Anchor Synthesizer Based on Seq2Seq and GAN Models
Authors:
- 1 October 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 233-236
Abstract
This paper presents a novel framework to generate realistic face video of an anchor, who is reading certain news. This task is also known as Virtual Anchor. Given some paragraphs of words, we first utilize a pretrained Word2Vec model to embed each word into a vector; then we utilize a Seq2Seq-based model to translate these word embeddings into action units and head poses of the target anchor; these action units and head poses will be concatenated with facial landmarks as well as the former n synthesized frames, and the concatenation serves as input of a Pix2PixHD-based model to synthesize realistic facial images for the virtual anchor. The experimental results demonstrate our framework is feasible for the synthesis of virtual anchor.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Photorealistic Facial Texture Inference Using Deep Neural Networks2017
- Image-to-Image Translation with Conditional Adversarial Networks2017
- Pinscreen2016
- Demo of Face2Face2016
- Deep Residual Learning for Image Recognition2016
- Dynamic 3D avatar creation from hand-held video inputACM Transactions on Graphics, 2015
- End-to-end learning of semantic role labeling using recurrent neural networks2015
- Automatic Face Reenactment2014
- Glove: Global Vectors for Word Representation2014
- Long Short-Term MemoryNeural Computation, 1997