DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System
Authors:
- Ankur HandaAHAnkur Handa
- Karl Van WykKWKarl Van Wyk
- Wei YangWYWei Yang
- Jacky LiangJLJacky Liang
- Yu-Wei ChaoYCYu-Wei Chao
- Qian WanQWQian Wan
- Stan BirchfieldSBStan Birchfield
- Nathan RatliffNRNathan Ratliff
- Dieter FoxDFDieter Fox
Preprint
Abstract
Teleoperation offers the possibility of imparting robotic systems with sophisticated reasoning skills, intuition, and creativity to perform tasks. However, current teleoperation solutions for high degree-of-actuation (DoA), multi-fingered robots are generally cost-prohibitive, while low-cost offerings usually provide reduced degrees of control. Herein, a low-cost, vision based teleoperation system, DexPilot, was developed that allows for complete control over the full 23 DoA robotic system by merely observing the bare human hand. DexPilot enables operators to carry out a variety of complex manipulation tasks that go beyond simple pick-and-place operations. This allows for collection of high dimensional, multi-modality, state-action data that can be leveraged in the future to learn sensorimotor policies for challenging manipulation tasks. The system performance was measured through speed and reliability metrics across two human demonstrators on a variety of tasks. The videos of the experiments can be found at https://sites.google.com/view/dex-pilot.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- RMPflow: A Computational Graph for Automatic Motion Policy Generation2020
- Shared human–robot proportional control of a dexterous myoelectric prosthesisNature Machine Intelligence, 2019
- Learning the signatures of the human grasp using a scalable tactile gloveNature, 2019
- Point-to-Point Regression PointNet for 3D Hand Pose Estimation2018
- Deep Residual Learning for Image Recognition2016
- DART: dense articulated real-time tracking with consumer depth camerasAutonomous Robots, 2015
- Coding and use of tactile signals from the fingertips in object manipulation tasksNature Reviews Neuroscience, 2009
- Transcontinental Telesurgical Nephrectomy Using the da Vinci Robot in a Porcine ModelUrology, 2008
- The multivariate L 1 -median and associated data depthProceedings of the National Academy of Sciences, 2000
- Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objectsExperimental Brain Research, 1984