Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control
Preprint
Abstract
This paper introduces a new generative deep learning network for human motion synthesis and control. Our key idea is to combine recurrent neural networks (RNNs) and adversarial training for human motion modeling. We first describe an efficient method for training a RNNs model from prerecorded motion data. We implement recurrent neural networks with long short-term memory (LSTM) cells because they are capable of handling nonlinear dynamics and long term temporal dependencies present in human motions. Next, we train a refiner network using an adversarial loss, similar to Generative Adversarial Networks (GANs), such that the refined motion sequences are indistinguishable from real motion capture data using a discriminative network. We embed contact information into the generative deep learning model to further improve the performance of our generative model. The resulting model is appealing to motion synthesis and control because it is compact, contact-aware, and can generate an infinite number of naturally looking motions with infinite lengths. Our experiments show that motions generated by our deep learning model are always highly realistic and comparable to high-quality motion capture data. We demonstrate the power and effectiveness of our models by exploring a variety of applications, ranging from random motion synthesis, online/offline motion control, and motion filtering. We show the superiority of our generative model by comparison against baseline models.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Learning from Simulated and Unsupervised Images through Adversarial Training2017
- On Human Motion Prediction Using Recurrent Neural Networks2017
- Deep Representation Learning for Human Motion Prediction and Classification2017
- Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network2017
- 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction2016
- Structural-RNN: Deep Learning on Spatio-Temporal Graphs2016
- Interactive generation of human animation with deformable motion modelsACM Transactions on Graphics, 2009
- Constraint-based motion optimization using a statistical dynamic modelACM Transactions on Graphics, 2007
- On the limited memory BFGS method for large scale optimizationMathematical Programming, 1989
- Generalization of backpropagation with application to a recurrent gas market modelNeural Networks, 1988