Grasp Pose Generation for Human-to-Robot Handovers Using Simulation-to-Reality Transfer
Open Access
- 21 February 2026
- book chapter
- Published by Springer Nature
- p. 215-222
Abstract
Human-to-robot handovers play an important role in collaborative tasks in industry or household assistance. Due to the vast amount of possible unknown objects, learning-based approaches gained interest for robust and general grasp synthesis. However, obtaining real training data for such methods requires expensive human demonstrations. Simulated data, on the other hand, is easy to generate and can be randomized to cover the distribution of real-world data. The first contribution of this work is a dataset for human grasps generated in simulation. For this, we use a simulated hand and models of 10 objects from the YCB dataset [1]. It can also be easily extended to include new objects. The method thus allows for generating an arbitrary amount of training data without human interactions. Secondly, we combine a generative neural grasp generator with an evaluator model for grasp pose generation. In contrast to previous works, we obtain grasp poses from simulated RGB images which allows for reducing the negative effects of depth sensor noise. To this end, our generator model is provided with a cropped image of the human hand and learns the distribution of grasps in the wrist system. The evaluator then narrows down the list of grasps to the most promising ones. The presented approach requires the model to extract relevant features from images instead of point clouds. A cost-efficient method for generating large amounts of training data is therefore needed. We test our approach in simulation and transfer it to a real robot system. We use the same objects as in the training dataset but also test the generalization capabilities toward new objects. The presented dataset is available for download: https://tuc.cloud/index.php/s/g3noZD7oCqbQR9d .Keywords
This publication has 21 references indexed in Scilit:
- HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers2022
- DexYCB: A Benchmark for Capturing Hand Grasping of Objects2021
- 6-DOF GraspNet: Variational Grasp Generation for Object Manipulation2019
- On the Continuity of Rotation Representations in Neural Networks2019
- Learning Sampling Distributions for Robot Motion Planning2018
- Hand in Hand with Robots: Differences Between Experienced and Naive Users in Human-Robot Handover Scenarios2017
- A human-inspired controller for fluid human-robot handovers2016
- Deep Residual Learning for Image Recognition2016
- The YCB object and Model set: Towards common benchmarks for manipulation research2015
- Microsoft COCO: Common Objects in Context2014