MeTRAbs: Metric-Scale Truncation-Robust Heatmaps for Absolute 3D Human Pose Estimation
Authors:
- István SárándiISIstván Sárándi
- Timm LinderTLTimm Linder
- Kai O. ArrasKAKai O. Arras
- Bastian LeibeBLBastian Leibe
Preprint
Abstract
Heatmap representations have formed the basis of human pose estimation systems for many years, and their extension to 3D has been a fruitful line of recent research. This includes 2.5D volumetric heatmaps, whose X and Y axes correspond to image space and Z to metric depth around the subject. To obtain metric-scale predictions, 2.5D methods need a separate post-processing step to resolve scale ambiguity. Further, they cannot localize body joints outside the image boundaries, leading to incomplete estimates for truncated images. To address these limitations, we propose metric-scale truncation-robust (MeTRo) volumetric heatmaps, whose dimensions are all defined in metric 3D space, instead of being aligned with image space. This reinterpretation of heatmap dimensions allows us to directly estimate complete, metric-scale poses without test-time knowledge of distance or relying on anthropometric heuristics, such as bone lengths. To further demonstrate the utility our representation, we present a differentiable combination of our 3D metric-scale heatmaps with 2D image-space ones to estimate absolute 3D pose (our MeTRAbs architecture). We find that supervision via absolute pose loss is crucial for accurate non-root-relative localization. Using a ResNet-50 backbone without further learned layers, we obtain state-of-the-art results on Human3.6M, MPI-INF-3DHP and MuPoTS-3D. Our code will be made publicly available to facilitate further research.Keywords
All Related Versions
This publication has 59 references indexed in Scilit:
- Multi-person Absolute 3D Human Pose Estimation with Weak Depth Supervision2020
- Beyond Weak Perspective for Monocular 3D Human Pose Estimation2020
- Human pose regression by combining indirect part detection and contextual informationComputers & Graphics, 2019
- Multi-Scale Structure-Aware Network for Human Pose Estimation2018
- Hand Pose Estimation via Latent 2.5D Heatmap Regression2018
- Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera2018
- Integral Human Pose Regression2018
- Identity Mappings in Deep Residual Networks2016
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search2008