Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registration
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 13269-13278
Abstract
Recent years have witnessed significant progress in 3D hand mesh recovery. Nevertheless, because of the intrinsic 2D-to-3D ambiguity, recovering camera-space 3D information from a single RGB image remains challenging. To tackle this problem, we divide camera-space mesh recovery into two sub-tasks, i.e., root-relative mesh recovery and root recovery. First, joint landmarks and silhouette are extracted from a single input image to provide 2D cues for the 3D tasks. In the root-relative mesh recovery task, we exploit semantic relations among joints to generate a 3D mesh from the extracted 2D cues. Such generated 3D mesh coordinates are expressed relative to a root position, i.e., wrist of the hand. In the root recovery task, the root position is registered to the camera space by aligning the generated 3D mesh back to 2D cues, thereby completing camera-space 3D mesh recovery. Our pipeline is novel in that (1) it explicitly makes use of known semantic relations among joints and (2) it exploits 1D projections of the silhouette and mesh to achieve robust registration. Extensive experiments on popular datasets such as FreiHAND, RHD, and Human3.6M demonstrate that our approach achieves state-of-the-art performance on both root-relative mesh recovery and root recovery. Our code is publicly available at https://github.com/SeanChenxy/HandMesh.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose2017
- An Efficient Algebraic Solution to the Perspective-Three-Point Problem2017
- LCR-Net: Localization-Classification-Regression for Human Pose2017
- Deep Residual Learning for Image Recognition2016
- SMPLACM Transactions on Graphics, 2015
- Going deeper with convolutions2015
- Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural EnvironmentsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Sequential Quadratic ProgrammingActa Numerica, 1995
- A Computational Approach to Edge DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 1986
- Generalized Procrustes AnalysisPsychometrika, 1975