Hand-Object Contact Consistency Reasoning for Human Grasps Generation
Authors:
Top Cited Papers
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 11087-11096
Abstract
While predicting robot grasps with parallel jaw grippers have been well studied and widely applied in robot manipulation tasks, the study on natural human grasp generation with a multi-finger hand remains a very challenging problem. In this paper, we propose to generate human grasps given a 3D object in the world. Our key observation is that it is crucial to model the consistency between the hand contact points and object contact regions. That is, we encourage the prior hand contact points to be close to the object surface and the object common contact regions to be touched by the hand at the same time. Based on the hand-object contact consistency, we design novel objectives in training the human grasp generation model and also a new self-supervised task which allows the grasp generation network to be adjusted even during test time. Our experiments show significant improvement in human grasp generation over state-of-the-art approaches by a large margin. More interestingly, by optimizing the model during test time with the self-supervised task, it helps achieve larger gain on unseen and out-of-domain objects. Project page: https://hwjiang1510.github.io/GraspTTA/.Keywords
All Related Versions
This publication has 38 references indexed in Scilit:
- A Comprehensive Survey on Domain Adaptation for Visual Applications2017
- Hand-Object Interaction Detection with Fully Convolutional Networks2017
- A compact representation of human single-object grasping2016
- Capturing Hands in Action Using Discriminative Salient Points and Physics SimulationInternational Journal of Computer Vision, 2016
- Understanding Everyday Hands in Action from RGB-D Images2015
- Grasp type revisited: A modern perspective on a classical feature for vision2015
- Tracking-Learning-DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2011
- Online domain adaptation of a pre-trained cascade of classifiers2011
- GraspIt!IEEE Robotics & Automation Magazine, 2004
- A hand-pose estimation for vision-based human interfacesIEEE Transactions on Industrial Electronics, 2003