Topological Planning with Transformers for Vision-and-Language Navigation
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 11271-11281
Abstract
Conventional approaches to vision-and-language navigation (VLN) are trained end-to-end but struggle to perform well in freely traversable environments. Inspired by the robotics community, we propose a modular approach to VLN using topological maps. Given a natural language instruction and topological map, our approach leverages attention mechanisms to predict a navigation plan in the map. The plan is then executed with low-level actions (e.g. FORWARD, ROTATE) using a robust controller. Experiments show that our method outperforms previous end-to-end approaches, generates interpretable navigation plans, and exhibits intelligent behaviors such as backtracking.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout2019
- Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation2018
- Cognitive Mapping and Planning for Visual Navigation2017
- Target-driven visual navigation in indoor scenes using deep reinforcement learning2017
- Deep Residual Learning for Image Recognition2016
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Following directions using statistical machine translation2010
- Stanley: The robot that won the DARPA Grand ChallengeJournal of Field Robotics, 2006
- Learning metric-topological maps for indoor mobile robot navigationArtificial Intelligence, 1998
- Long Short-Term MemoryNeural Computation, 1997