Fast and Robust Hand Tracking Using Detection-Guided Optimization
Preprint
Abstract
Markerless tracking of hands and fingers is a promising enabler for human-computer interaction. However, adoption has been limited because of tracking inaccuracies, incomplete coverage of motions, low framerate, complex camera setups, and high computational requirements. In this paper, we present a fast method for accurately tracking rapid and complex articulations of the hand using a single depth camera. Our algorithm uses a novel detection-guided optimization strategy that increases the robustness and speed of pose estimation. In the detection step, a randomized decision forest classifies pixels into parts of the hand. In the optimization step, a novel objective function combines the detected part labels and a Gaussian mixture representation of the depth to estimate a pose that best fits the depth. Our approach needs comparably less computational resources which makes it extremely fast (50 fps without GPU support). The approach also supports varying static, or moving, camera-to-scene arrangements. We show the benefits of our method by evaluating on public datasets and comparing against previous work.Keywords
All Related Versions
- Version 1, 2016-02-12, ArXiv
- Published version: (10636919), 3213.
This publication has 10 references indexed in Scilit:
- Real-Time Continuous Pose Recovery of Human Hands Using Convolutional NetworksACM Transactions on Graphics, 2014
- Learning to be a depth camera for close-range human capture and interactionACM Transactions on Graphics, 2014
- Real-time non-rigid reconstruction using an RGB-D cameraACM Transactions on Graphics, 2014
- Real-Time Articulated Hand Pose Estimation Using Semi-supervised Transductive Regression Forests2013
- Video-based hand manipulation capture through composite motion controlACM Transactions on Graphics, 2013
- A data-driven approach for real-time full body pose reconstruction from a depth camera2011
- Full DOF tracking of a hand interacting with an object by modeling occlusions and physical constraints2011
- A survey of glove-based inputIEEE Computer Graphics and Applications, 1994
- A hand gesture interface deviceACM SIGCHI Bulletin, 1986
- Capturing Hand Motion with an RGB-D Sensor, Fusing a Generative Model with Salient Points