Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model
Authors:
Top Cited Papers
- 1 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 481-490
Abstract
With the recent surge in popularity of AR/VR applications, realistic and accurate control of 3D full-body avatars has become a highly demanded feature. A particular challenge is that only a sparse tracking signal is available from standalone HMDs (Head Mounted Devices), often limited to tracking the user's head and wrists. While this signal is resourceful for reconstructing the upper body motion, the lower body is not tracked and must be synthesized from the limited information provided by the upper body joints. In this paper, we present AGRoL, a novel conditional diffusion model specifically designed to track full bodies given sparse upper-body tracking signals. Our model is based on a simple multi-layer perceptron (MLP) architecture and a novel conditioning scheme for motion data. It can predict accurate and smooth full-body motion, particularly the challenging lower body movement. Unlike common diffusion architectures, our compact architecture can run in real-time, making it suitable for online body-tracking applications. We train and evaluate our model on AMASS motion capture dataset, and demonstrate that our approach outperforms state-of-the-art methods in generated motion accuracy and smoothness. We further justify our design choices through extensive experiments and ablation studies.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Deep Residual Learning for Image Recognition2016
- Structural-RNN: Deep Learning on Spatio-Temporal Graphs2016
- Recurrent Network Models for Human Dynamics2015
- SMPLACM Transactions on Graphics, 2015
- The KIT whole-body human motion database2015
- Pose-conditioned joint angle limits for 3D human pose reconstruction2015
- MoShACM Transactions on Graphics, 2014
- HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human MotionInternational Journal of Computer Vision, 2009
- Decomposing biological motion: A framework for analysis and synthesis of human gait patternsJournal of Vision, 2002
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989