Transformer Memory for Interactive Visual Navigation in Cluttered Environments
- 2 February 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (3) , 1731-1738
Abstract
Substantial progress has been achieved in embodied visual navigation based on reinforcement learning (RL). These studies presume that the environment is stationary where all the obstacles are static. However, in real cluttered scenes, interactable objects (e.g. shoes and boxes) blocking the way of robots makes the environment non-stationary. Accordingly, the ego-centric visual agent will easily get stuck in the dilemma of finding the next waypoint as it struggles to decide whether to push the obstacles ahead. To handle the predicament, we formulate this interactive visual navigation as Partial Observed Markov Decision Process (POMDP). As the transformer encoder has demonstrated its superior ability to capture the spatial-temporal dependencies in natural language processing. We propose a transformer-based memory to empower the agents utilizing the historical interactive information. However, directly leveraging the transformer architecture in the RL settings is highly unstable. We further propose a surrogate objective to predict the next waypoint as the auxiliary task, which facilitates the representation learning and bootstraps the RL. We demonstrate our method in the iGibson environment and experimental results show a significant improvement over the interactive Gibson benchmark and the related recurrent RL policy both in the validation seen scenes and the test unseen scenes.Keywords
Funding Information
- Science and Technology Commission of Shanghai Municipality (22DZ2229004)
- Shanghai Trusted Industry Internet Software Collaborative Innovation Center
This publication has 15 references indexed in Scilit:
- iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes2021
- CrossMap Transformer: A Crossmodal Masked Path Transformer Using Double Back-Translation for Vision-and-Language NavigationIEEE Robotics and Automation Letters, 2021
- Semantic Audio-Visual Navigation2021
- Pushing it out of the Way: Interactive Visual Navigation2021
- Visual Navigation Among Humans With Optimal Control as a SupervisorIEEE Robotics and Automation Letters, 2021
- Interactive Gibson Benchmark: A Benchmark for Interactive Navigation in Cluttered EnvironmentsIEEE Robotics and Automation Letters, 2020
- Habitat: A Platform for Embodied AI Research2019
- Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks2019
- Cognitive Mapping and Planning for Visual Navigation2017
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014