RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking
Authors:
Top Cited Papers
- 13 May 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4788-4795
Abstract
The grand aim of having a single robot that can manipulate arbitrary objects in diverse settings is at odds with the paucity of robotics datasets. Acquiring and growing such datasets is strenuous due to manual efforts, operational costs, and safety challenges. A path toward such a universal agent requires an efficient framework capable of generalization but within a reasonable data budget. In this paper, we develop an efficient framework (MT-ACT) for training universal agents capable of multi-task manipulation skills using (a) semantic augmentations that can rapidly multiply existing datasets and (b) action representations that can extract performant policies with small yet diverse multi-modal datasets without overfitting. In addition, reliable task conditioning and an expressive policy architecture enables our agent to exhibit a diverse repertoire of skills in novel situations specified using task commands. Using merely 7500 demonstrations, we are able to train a single policy RoboAgent capable of 12 unique skills, and demonstrate its generalization over 38 tasks spread across common daily activities in diverse kitchen scenes. On average, RoboAgent outperforms prior methods by over 40% in unseen situations while being more sample efficient. See https://robopen.github.io/for video results and appendix.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- Segment Anything2023
- Affordances from Human Videos as a Versatile Representation for Robotics2023
- Ego4D: Around the World in 3,000 Hours of Egocentric Video2022
- Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention2021
- Mastering Atari, Go, chess and shogi by planning with a learned modelNature, 2020
- RL-CycleGAN: Reinforcement Learning Aware Simulation-to-Real2020
- Improving Data Efficiency of Self-supervised Learning for Robotic Grasping2019
- Translating Videos to Commands for Robotic Manipulation with Deep Recurrent Neural Networks2018
- Learning to push by grasping: Using multiple tasks for effective learning2017
- ImageNet: A large-scale hierarchical image database2009