Uncertainty-Aware Testing-Time Optimization for 3D Human Pose Estimation
- 12 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 2837-2849
Abstract
Although data-driven methods have achieved success in 3D human pose estimation, they often suffer from domain gaps and exhibit limited generalization. In contrast, optimization-based methods excel in fine-tuning for specific cases but are generally inferior to data-driven methods in overall performance. We observe that previous optimization-based methods commonly rely on projection constraint, which only ensures alignment in 2D space, potentially leading to the overfitting problem. To address this, we propose an Uncertainty-Aware testing-time Optimization (UAO) framework, which keeps the prior information of pre-trained model and alleviates the overfitting problem using the uncertainty of joints. Specifically, during the training phase, we design an effective 2D-to-3D network for estimating the corresponding 3D pose while quantifying the uncertainty of each 3D joint. For optimization during testing, the proposed optimization framework freezes the pre-trained model and optimizes only a latent state. Projection loss is then employed to ensure the generated poses are well aligned in 2D space for high-quality optimization. Furthermore, we utilize the uncertainty of each joint to determine how much each joint is allowed for optimization. The effectiveness and superiority of the proposed framework are validated through extensive experiments on challenging datasets: Human3.6M, MPI-INF-3DHP, and 3DPW. Notably, our approach outperforms the previous best result by a large margin of 5.5% on Human3.6M.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62473007)
- Shenzhen Innovation in Science and Technology Foundation for The Excellent Youth Scholars (RCYX20231211090248064)
This publication has 71 references indexed in Scilit:
- Learning Enriched Hop-Aware Correlation for Robust 3D Human Pose EstimationInternational Journal of Computer Vision, 2023
- SRNet: Improving Generalization in 3D Human Pose Estimation with a Split-and-Recombine Approach2020
- Predicting Camera Viewpoint Improves Cross-Dataset Generalization for 3D Human Pose Estimation2020
- Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera2018
- Integral Human Pose Regression2018
- Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- A survey of human pose estimation: The body parts parsing based methodsJournal of Visual Communication and Image Representation, 2015
- Reconstructing 3D Human Pose from 2D Image Landmarks2012
- Generalized Procrustes AnalysisPsychometrika, 1975