HYRE: Hybrid Regressor for 3D Human Pose and Shape Estimation
- 25 December 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34 (10577149) , 235-246
Abstract
Regression-based 3D human pose and shape estimation often fall into one of two different paradigms. Parametric approaches, which regress the parameters of a human body model, tend to produce physically plausible but image-mesh misalignment results. In contrast, non-parametric approaches directly regress human mesh vertices, resulting in pixel-aligned but unreasonable predictions. In this paper, we consider these two paradigms together for a better overall estimation. To this end, we propose a novel HYbrid REgressor (HYRE) that greatly benefits from the joint learning of both paradigms. The core of our HYRE is a hybrid intermediary across paradigms that provides complementary clues to each paradigm at the shared feature level and fuses their results at the part-based decision level, thereby bridging the gap between the two. We demonstrate the effectiveness of the proposed method through both quantitative and qualitative experimental analyses, resulting in improvements for each approach and ultimately leading to better hybrid results. Our experiments show that HYRE outperforms previous methods on challenging 3D human pose and shape benchmarks.Keywords
Funding Information
- National Natural Science Foundation of China (62373009)
- Natural Science Foundation of Guangdong Province (2024A1515012089)
- Natural Science Foundation of Shenzhen (JCYJ20230807120801002)
- Shenzhen Innovation in Science and Technology Foundation for The Excellent Youth Scholars (RCYX20231211090248064)
- Interdisciplinary Doctoral Grant from the Personalized Health and Related Technologies (PHRT) of ETH Domain, Switzerland (iDoc 2021-360)
- European Union’s Horizon Europe research and innovation program (101120237 (ELIAS))
- PNRR Project Future AI Research
- NRRP MUR program funded by the NextGenerationEU (PE00000013)
This publication has 63 references indexed in Scilit:
- POCO: 3D Pose and Shape Estimation with Confidence2024
- CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation2022
- D &D: Learning Human Dynamics from Dynamic Camera2022
- Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers2022
- Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation2021
- Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera2018
- Towards Accurate Marker-Less Human Shape and Pose Estimation over Time2017
- Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision2017
- Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image2016
- 2D Human Pose Estimation: New Benchmark and State of the Art Analysis2014