Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale
Authors:
Top Cited Papers
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5163-5173
Abstract
We present a large-scale study of imitating human demonstrations on tasks that require a virtual robot to search for objects in new environments - (1) ObjectGoal Navigation (e.g. 'find & go to a chair’) and (2) Pick&place (e.g. 'find mug, pick mug, find counter, place mug on counter’). First, we develop a virtual teleoperation data-collection infrastructure - connecting Habitat simulator running in a web browser to Amazon Mechanical Turk, allowing remote users to teleoperate virtual robots, safely and at scale. We collect 80k demonstrations for OBJECTNAV and 12k demonstrations for PICK&PLACE, which is an order of magnitude larger than existing human demonstration datasets in simulation or on real robots. Our virtual teleoperation data contains 29.3M actions, and is equivalent to 22.6k hours of real-world teleoperation time, and illustrates rich, diverse strategies for solving the tasks. Second, we use this data to answer the question - how does large-scale imitation learning (IL) (which has not been hitherto possible) compare to reinforcement learning (RL) (which is the status quo)? On OBJECTNAV, we find that IL (with no bells or whistles) using 70k human demonstrations outperforms RL using 240k agent-gathered trajectories. This effectively establishes an ‘exchange rate’ - a single human demonstration appears to be worth ~4 agent-gathered ones. More importantly, we find the IL-trained agent learns efficient object-search behavior from humans - it peeks into rooms, checks corners for small objects, turns in place to get a panoramic view - none of these are exhibited as prominently by the RL agent, and to induce these behaviors via contemporary RL techniques would require tedious reward engineering. Finally, accuracy vs. training data size plots show promising scaling behavior, suggesting that simply collecting more demonstrations is likely to advance the state of art further. On PICK&PLACE, the comparison is starker - IL agents achieve ~18% success on episodes with new object-receptacle locations when trained with 9.5k human demonstrations, while RL agents fail to get beyond 0%. Overall, our work provides compelling evidence for investing in large-scale imitation learning.Keywords
All Related Versions
Funding Information
- NSF
- ONR
This publication has 18 references indexed in Scilit:
- Gibson Env: Real-World Perception for Embodied Agents2018
- IQA: Visual Question Answering in Interactive Environments2018
- Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments2018
- Matterport3D: Learning from RGB-D Data in Indoor Environments2017
- Curiosity-Driven Exploration by Self-Supervised Prediction2017
- Target-driven visual navigation in indoor scenes using deep reinforcement learning2017
- Deep Residual Learning for Image Recognition2016
- SUN RGB-D: A RGB-D scene understanding benchmark suite2015
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Long Short-Term MemoryNeural Computation, 1997