DexTOG: Learning Task-Oriented Dexterous Grasp With Language Condition
- 16 December 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 10 (2) , 995-1002
Abstract
This study introduces a novel language-guided diffusion-based learning framework, DexTOG, aimed at advancing the field of task-oriented grasping (TOG) with dexterous hands. Unlike existing methods that mainly focus on 2-finger grippers, this research addresses the complexities of dexterous manipulation, where the system must identify non-unique optimal grasp poses under specific task constraints, cater to multiple valid grasps, and search in a high degree-of-freedom configuration space in grasp planning. The proposed DexTOG includes a diffusion-based grasp pose generation model, DexDiffu, and a data engine to support the DexDiffu. By leveraging DexTOG, we also proposed a new dataset, DexTOG-80K, which was developed using a shadow robot hand to perform various tasks on 80 objects from 5 categories, showcasing the dexterity and multitasking capabilities of the robotic hand. This research not only presents a significant leap in dexterous TOG but also provides a comprehensive dataset and simulation validation, setting a new benchmark in robotic manipulation research.Keywords
All Related Versions
Funding Information
- National Key Research and Development Project of China (2022ZD0160102)
- National Key Research and Development Project of China (2021ZD0110704)
- Shanghai Artificial Intelligence Laboratory, XPLORER PRIZE
This publication has 27 references indexed in Scilit:
- UGG: Unified Generative Grasping2024
- Single-View Scene Point Cloud Human Grasp Generation2024
- UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy2023
- OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction2022
- Hand-Object Contact Consistency Reasoning for Human Grasps Generation2021
- DexYCB: A Benchmark for Capturing Hand Grasping of Objects2021
- ACRONYM: A Large-Scale Grasp Dataset Based on Simulation2021
- Learning Task-Oriented Dexterous Grasping from Human Knowledge2021
- On the Continuity of Rotation Representations in Neural Networks2019
- Learning Joint Reconstruction of Hands and Manipulated Objects2019