Universal Actions for Enhanced Embodied Foundation Models
Authors:
- Jianxiong LiJLJianxiong Li
- Yinan ZhengYZYinan Zheng
- Xianyuan ZhanXZXianyuan Zhan
Preprint
Abstract
Training on diverse, internet-scale data is a key factor in the success of recent large foundation models. Yet, using the same recipe for building embodied agents has faced noticeable difficulties. Despite the availability of many crowd-sourced embodied datasets, their action spaces often exhibit significant heterogeneity due to distinct physical embodiment and control interfaces for different robots, causing substantial challenges in developing embodied foundation models using cross-domain data. In this paper, we introduce UniAct, a new embodied foundation modeling framework operating in a tokenized Universal Action Space. Our learned universal actions capture the generic atomic behaviors across diverse robots by exploiting their shared structural features, and enable enhanced cross-domain data utilization and cross-embodiment generalizations by eliminating the notorious heterogeneity. The universal actions can be efficiently translated back to heterogeneous actionable commands by simply adding embodiment-specific details, from which fast adaptation to new robots becomes simple and straightforward. Our 0.5B instantiation of UniAct outperforms 14X larger SOTA embodied foundation models in extensive evaluations on various real-world and simulation robots, showcasing exceptional cross-embodiment control and adaptation capability, highlighting the crucial benefit of adopting universal actions. Project page: https://github.com/2toinf/UniActKeywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning2025
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset2024
- Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation2024
- Any-point Trajectory Modeling for Policy Learning2024
- Yell At Your Robot: Improving On-the-Fly from Language Corrections2024
- Octo: An Open-Source Generalist Robot Policy2024
- Language-Driven Representation Learning for Robotics2023
- RT-1: Robotics Transformer for Real-World Control at Scale2023
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion2023
- Towards artificial general intelligence via a multimodal foundation modelNature Communications, 2022