UniGraspTransformer: Simplified Policy Distillation for Scalable Dexterous Robotic Grasping

Authors:
  • WWWenbo Wang
    Wenbo Wang
  • FWFangyun Wei
    Fangyun Wei
  • LZLei Zhou
    Lei Zhou
  • XCXi Chen
    Xi Chen
  • LLLin Luo
    Lin Luo
  • XYXiaohan Yi
    Xiaohan Yi
  • YZYizhong Zhang
    Yizhong Zhang
  • YLYaobo Liang
    Yaobo Liang
  • CXChang Xu
    Chang Xu
  • YLYan Lu
    Yan Lu
  • JYJiaolong Yang
    Jiaolong Yang
  • BGBaining Guo
    Baining Guo

Abstract
We introduce UniGraspTransformer, a universal Transformer-based network for dexterous robotic grasping that simplifies training while enhancing scalability and performance. Unlike prior methods such as UniDexGrasp++, which require complex, multi-step training pipelines, UniGraspTransformer follows a streamlined process: first, dedicated policy networks are trained for individual objects using reinforcement learning to generate successful grasp trajectories; then, these trajectories are distilled into a single, universal network. Our approach enables UniGraspTransformer to scale effectively, incorporating up to 12 self-attention blocks for handling thousands of objects with diverse poses. Additionally, it generalizes well to both idealized and real-world inputs, evaluated in state-based and vision-based settings. Notably, UniGraspTransformer generates a broader range of grasping poses for objects in various shapes and orientations, resulting in more diverse grasp strategies. Experimental results demonstrate significant improvements over state-of-the-art, UniDexGrasp++, across various object categories, achieving success rate gains of 3.5%, 7.7%, and 10.1% on seen objects, unseen objects within seen categories, and completely unseen objects, respectively, in the vision-based setting. Project page: https://dexhand.github.io/UniGraspTransformer.