DeFeeNet: Consecutive 3D Human Motion Prediction with Deviation Feedback
Authors:
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5527-5536
Abstract
Let us rethink the real-world scenarios that require human motion prediction techniques, such as human-robot collaboration. Current works simplify the task of predicting human motions into a one-off process of forecasting a short future sequence (usually no longer than 1 second) based on a historical observed one. However, such simplification may fail to meet practical needs due to the neglect of the fact that motion prediction in real applications is not an isolated “observe then predict” unit, but a consecutive process composed of many rounds of such unit, semi-overlapped along the entire sequence. As time goes on, the predicted part of previous round has its corresponding ground truth observable in the new round, but their deviation in-between is neither exploited nor able to be captured by existing isolated learning fashion. In this paper, we propose DeFeeNet, a simple yet effective network that can be added on existing one-off prediction models to realize deviation perception and feedback when applied to consecutive motion prediction task. At each prediction round, the deviation generated by previous unit is first encoded by our DeFeeNet, and then incorporated into the existing predictor to enable a deviation-aware prediction manner, which, for the first time, allows for information transmit across adjacent prediction units. We design two versions of DeFeeNet as MLP-based and GRU-based, respectively. On Human3.6M and more complicated BABEL, experimental results indicate that our proposed network improves consecutive human motion prediction performance regardless of the basic model.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62176125,61772272)
This publication has 28 references indexed in Scilit:
- Context-Aware Human Motion Prediction2020
- Modeling Human Motion with Quaternion-Based Neural NetworksInternational Journal of Computer Vision, 2019
- Learning Trajectory Dependencies for Human Motion Prediction2019
- Dest-ResNet2018
- Convolutional Sequence to Sequence Model for Human Dynamics2018
- On Human Motion Prediction Using Recurrent Neural Networks2017
- Structural-RNN: Deep Learning on Spatio-Temporal Graphs2016
- Recurrent Network Models for Human Dynamics2015
- SMPLACM Transactions on Graphics, 2015
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013