Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregation
Authors:
Top Cited Papers
- 1 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 14715-14725
Abstract
In this paper, a novel Diffusion-based 3D Pose estimation (D3DP) method with Joint-wise reProjection-based Multi-hypothesis Aggregation (JPMA) is proposed for probabilistic 3D human pose estimation. On the one hand, D3DP generates multiple possible 3D pose hypotheses for a single 2D observation. It gradually diffuses the ground truth 3D poses to a random distribution, and learns a denoiser conditioned on 2D keypoints to recover the uncontaminated 3D poses. The proposed D3DP is compatible with existing 3D pose estimators and supports users to balance efficiency and accuracy during inference through two customizable parameters. On the other hand, JPMA is proposed to assemble multiple hypotheses generated by D3DP into a single 3D pose for practical use. It reprojects 3D pose hypotheses to the 2D camera plane, selects the best hypothesis joint-by-joint based on the reprojection errors, and combines the selected joints into the final pose. The proposed JPMA conducts aggregation at the joint level and makes use of the 2D prior information, both of which have been overlooked by previous approaches. Extensive experiments on Human3.6M and MPI-INF-3DHP datasets show that our method outperforms the state-of-the-art deterministic and probabilistic approaches by 1.5% and 8.9%, respectively. Code is available at https://github.com/paTRICK-swk/D3DP.Keywords
All Related Versions
This publication has 43 references indexed in Scilit:
- P-STMO: Pre-trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimation2022
- Motion Guided 3D Pose Estimation from Videos2020
- SRNet: Improving Generalization in 3D Human Pose Estimation with a Split-and-Recombine Approach2020
- Unsupervised 3D Pose Estimation With Geometric Self-Supervision2019
- Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera2018
- Cascaded Pyramid Network for Multi-person Pose Estimation2018
- Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision2017
- Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields2017
- Single image 3D human pose estimation from noisy observations2012
- Proposal maps driven MCMC for estimating human body pose in static images2004