Using a single RGB frame for real time 3D hand pose estimation in the wild
Authors:
Preprint
Abstract
We present a method for the real-time estimation of the full 3D pose of one or more human hands using a single commodity RGB camera. Recent work in the area has displayed impressive progress using RGBD input. However, since the introduction of RGBD sensors, there has been little progress for the case of monocular color input. We capitalize on the latest advancements of deep learning, combining them with the power of generative hand pose estimation techniques to achieve real-time monocular 3D hand pose estimation in unrestricted scenarios. More specifically, given an RGB image and the relevant camera calibration information, we employ a state-of-the-art detector to localize hands. Given a crop of a hand in the image, we run the pretrained network of OpenPose for hands to estimate the 2D location of hand joints. Finally, non-linear least-squares minimization fits a 3D model of the hand to the estimated 2D joint positions, recovering the 3D hand pose. Extensive experimental results provide comparison to the state of the art as well as qualitative assessment of the method in the wild.Keywords
All Related Versions
This publication has 38 references indexed in Scilit:
- Large-scale multiview 3D hand pose datasetImage and Vision Computing, 2019
- Region ensemble network: Towards good practices for deep 3D hand pose estimationJournal of Visual Communication and Image Representation, 2018
- Capturing Hands in Action Using Discriminative Salient Points and Physics SimulationInternational Journal of Computer Vision, 2016
- Real-Time Hand Tracking Using a Sum of Anisotropic Gaussians Model2014
- Microsoft COCO: Common Objects in Context2014
- Hand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests2012
- Markerless and Efficient 26-DOF Hand Pose Recovery2011
- Vision-based hand pose estimation: A reviewComputer Vision and Image Understanding, 2007
- Model-based 3D tracking of an articulated hand2005
- Visual tracking of high DOF articulated structures: An application to human hand tracking1994