DexH2R: Task-Oriented Dexterous Manipulation From Human to Robots
- 29 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ASME Transactions on Mechatronics
- Vol. 31 (3) , 3202-3213
Abstract
Dexterous manipulation remains a challenging problem due to the high-dimensional action space of robot hands. To solve this challenge, recent advances focus on transferring human hand motions to dexterous robot motions. However, the most straightforward method, retargeting, fails to ensure successful task execution due to the ignorance of hand–object interaction and environmental feedback, while other methods combining retargeting with human feedback significantly increase human labor and degrade motion quality. To address this limitation, we propose a novel framework that enhances retargeted motions with a residual policy based on reinforcement learning, allowing robots to autonomously correct action errors without human intervention. Specifically, our approach leverages human demonstrations to provide natural and task-relevant finger motions while incorporating object and environmental information to improve manipulation performance. Extensive experiments in multiple robot hand embodiments demonstrate that our method outperforms baseline approaches by approximately 40% in task success rates on both seen and unseen objects. Furthermore, our approach eliminates the need for real-time human correction, significantly reducing labor costs and facilitating scalable data collection for dexterous manipulation.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning2025
- Physically Plausible Full-Body Hand-Object Interaction Synthesis2024
- UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning2023
- DexMV: Imitation Learning for Dexterous Manipulation from Human Videos2022
- GraspLook: a VR-based Telemanipulation System with R-CNN-driven Augmentation of Virtual Environment2021
- DexYCB: A Benchmark for Capturing Hand Grasping of Objects2021
- Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation2019
- Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation2018
- The YCB object and Model set: Towards common benchmarks for manipulation research2015
- A survey of robot learning from demonstrationRobotics and Autonomous Systems, 2009