FLAG: Flow-based 3D Avatar Generation from Sparse Observations
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 13243-13252
Abstract
To represent people in mixed reality applications for collaboration and communication, we need to generate realistic and faithful avatar poses. However, the signal streams that can be applied for this task from head-mounted devices (HMDs) are typically limited to head pose and hand pose estimates. While these signals are valuable, they are an incomplete representation of the human body, making it challenging to generate a faithful full-body avatar. We address this challenge by developing a flow-based generative model of the 3D human body from sparse observations, wherein we learn not only a conditional distribution of 3D human pose, but also a probabilistic mapping from observations to the latent space from which we can generate a plausible pose along with uncertainty estimates for the joints. We show that our approach is not only a strong predictive model, but can also act as an efficient pose prior in different optimization settings where a good initial latent code plays a major role.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- STAR: Sparse Trained Articulated Human Body Regressor2020
- PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization2020
- A Stochastic Conditioning Scheme for Diverse Human Motion Prediction2020
- Monocular 3D Human Pose Estimation by Generation and Ordinal Ranking2019
- On the Continuity of Rotation Representations in Neural Networks2019
- Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondencesACM Transactions on Graphics, 2016
- SMPLACM Transactions on Graphics, 2015
- MoShACM Transactions on Graphics, 2014
- HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human MotionInternational Journal of Computer Vision, 2009
- On the limited memory BFGS method for large scale optimizationMathematical Programming, 1989