FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration
Authors:
Top Cited Papers
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1749-1759
Abstract
3D human pose estimation can facilitate various applications, such as assistive technologies and AR/VR. Most existing monocular 3D pose estimation approaches only focus on a single body part, neglecting the fact that the essential nuance of human motion is conveyed through a concert of subtle movements of face, hands, and body. In this paper, we present FrankMocap, a fast and accurate whole-body 3D pose estimation system that can produce 3D face, hands, and body simultaneously from in-the-wild monocular images. The idea of FrankMocap is its modular design: We first run 3D pose regression methods for face, hands, and body independently, followed by composing the regression outputs via an integration module. The separate regression modules allow us to take full advantage of their state-of-the-art performances without compromising the original accuracy and reliability in practice. We develop three different integration modules that trade off between latency and accuracy. All of them are capable of providing simple yet effective solutions to unify the separate outputs into seamless whole-body pose estimation results. We quantitatively and qualitatively demonstrate that our modularized system outperforms both the optimization-based and end-to-end methods of estimating whole-body pose.Keywords
All Related Versions
This publication has 58 references indexed in Scilit:
- VNectACM Transactions on Graphics, 2017
- Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image2016
- Deep Residual Learning for Image Recognition2016
- SMPLACM Transactions on Graphics, 2015
- DynaACM Transactions on Graphics, 2015
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Modeling the Performance of Image Restoration From Motion BlurIEEE Transactions on Image Processing, 2012
- SCAPEACM Transactions on Graphics, 2005
- A morphable model for the synthesis of 3D faces1999
- Generalized Procrustes AnalysisPsychometrika, 1975