Multitask Non-Autoregressive Model for Human Motion Prediction
- 24 November 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 30 (10577149) , 2562-2574
Abstract
Human motion prediction, which aims at predicting future human skeletons given the past ones, is a typical sequence-to-sequence problem. Therefore, extensive efforts have been devoted to exploring different RNN-based encoder-decoder architectures. However, by generating target poses conditioned on the previously generated ones, these models are prone to bringing issues such as error accumulation problem. In this paper, we argue that such issue is mainly caused by adopting autoregressive manner. Hence, a novel Non-AuToregressive model (NAT) is proposed with a complete non-autoregressive decoding scheme, as well as a context encoder and a positional encoding module. More specifically, the context encoder embeds the given poses from temporal and spatial perspectives. The frame decoder is responsible for predicting each future pose independently. The positional encoding module injects positional signal into the model to indicate the temporal order. Besides, a multitask training paradigm is presented for both low-level human skeleton prediction and high-level human action recognition, resulting in the considerable improvement for the prediction task. Our approach is evaluated on Human3.6M and CMU-Mocap benchmarks and outperforms state-of-the-art autoregressive methods.Keywords
Funding Information
- Science and Technology Innovation 2030–“New Generation Artificial Intelligence” (2018AAA0100904)
- Zhejiang Provincial Natural Science Foundation of China (LR19F020004)
- National Natural Science Foundation of China (U20A20222)
- Key Scientific Technological Innovation Research Project by Ministry of Education, NSFC (61672456, 61702448, U19B2043)
- Artificial Intelligence Research Foundation of Baidu Inc., through the HIKVision and Horizon Robotics
- ZJU Converging Media Computing Lab
This publication has 35 references indexed in Scilit:
- A deep learning framework for character motion synthesis and editingACM Transactions on Graphics, 2016
- Structural-RNN: Deep Learning on Spatio-Temporal Graphs2016
- Unsupervised Visual Representation Learning by Context Prediction2015
- Recurrent Network Models for Human Dynamics2015
- On the Properties of Neural Machine Translation: Encoder–Decoder Approaches2014
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Practical Parameterization of Rotations Using the Exponential MapJournal of Graphics Tools, 1998
- Long Short-Term MemoryNeural Computation, 1997
- Nonlinear total variation based noise removal algorithmsPhysica D: Nonlinear Phenomena, 1992
- Animating rotation with quaternion curves1985