MeTRAbs: Metric-Scale Truncation-Robust Heatmaps for Absolute 3D Human Pose Estimation
- 11 November 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biometrics, Behavior, and Identity Science
- Vol. 3 (1) , 16-30
Abstract
Heatmap representations have formed the basis of human pose estimation systems for many years, and their extension to 3D has been a fruitful line of recent research. This includes 2.5D volumetric heatmaps, whose X and Y axes correspond to image space and Z to metric depth around the subject. To obtain metric-scale predictions, 2.5D methods need a separate post-processing step to resolve scale ambiguity. Further, they cannot localize body joints outside the image boundaries, leading to incomplete estimates for truncated images. To address these limitations, we propose metric-scale truncation-robust (MeTRo) volumetric heatmaps, whose dimensions are all defined in metric 3D space, instead of being aligned with image space. This reinterpretation of heatmap dimensions allows us to directly estimate complete, metric-scale poses without test-time knowledge of distance or relying on anthropometric heuristics, such as bone lengths. To further demonstrate the utility our representation, we present a differentiable combination of our 3D metric-scale heatmaps with 2D image-space ones to estimate absolute 3D pose (our MeTRAbs architecture). We find that supervision via absolute pose loss is crucial for accurate non-root-relative localization. Using a ResNet-50 backbone without further learned layers, we obtain state-of-the-art results on Human3.6M, MPI-INF-3DHP and MuPoTS-3D. Our code is publicly available.Keywords
All Related Versions
Funding Information
- Bosch Research Foundation
- European Research Council under Consolidator Grant Project DeeViSe (ERC-CoG-2017-773161)
- EU through the H2020 Projects (ILIAD H2020-ICT-2016b-732737, CROWDBOT H2020-ICT-2017-779942)
This publication has 51 references indexed in Scilit:
- VNectACM Transactions on Graphics, 2017
- Learning from Synthetic Humans2017
- LCR-Net: Localization-Classification-Regression for Human Pose2017
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- 3D Human pose estimation: A review of the literature and analysis of covariatesComputer Vision and Image Understanding, 2016
- Amodal Completion and Size Constancy in Natural Scenes2015
- DEEPFOCAL: A method for direct focal length estimation2015
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Latent structured models for human pose estimation2011