GNM: A General Navigation Model to Drive Any Robot
Authors:
Top Cited Papers
- 29 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7226-7233
Abstract
Learning provides a powerful tool for vision-based navigation, but the capabilities of learning-based policies are constrained by limited training data. If we could combine data from all available sources, including multiple kinds of robots, we could train more powerful navigation models. In this paper, we study how a general goal-conditioned model for vision-based navigation can be trained on data obtained from many distinct but structurally similar robots, and enable broad generalization across environments and embodiments. We analyze the necessary design decisions for effective data sharing across robots, including the use of temporal context and standardized action spaces, and demonstrate that an omnipolicy trained from heterogeneous datasets outperforms policies trained on any single dataset. We curate 60 hours of navigation trajectories from 6 distinct robots, and deploy the trained GNM on a range of new robots, including an underactuated quadrotor. We find that training on diverse data leads to robustness against degradation in sensing and actuation. Using a pre-trained navigation model with broad generalization capabilities can bootstrap applications on novel robots going forward, and we hope that the GNM represents a step in that direction. For more information on the datasets, code, and videos, please check out our project page 1 1 sites.google.com/view/drive-any-robot.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control2018
- Sim-to-Real Transfer of Robotic Control with Dynamics Randomization2018
- DroNet: Learning to Fly by DrivingIEEE Robotics and Automation Letters, 2018
- Preparing for the Unknown: Learning a Universal Policy with Online System Identification2017
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- Learning modular neural network policies for multi-task and multi-robot transfer2017
- University of Michigan North Campus long-term vision and lidar datasetThe International Journal of Robotics Research, 2015
- CNN Features Off-the-Shelf: An Astounding Baseline for Recognition2014
- Localization and navigation of the CoBots over long-term deploymentsThe International Journal of Robotics Research, 2013
- ImageNet: A large-scale hierarchical image database2009