Regularized Denoising Masked Visual Pretraining for Robust Embodied PointGoal Navigation
Open Access
Abstract
Embodied PointGoal navigation is a fundamental task for embodied agents. Recent works have shown that the performance of the embodied navigation agent degrades significantly in the presence of visual corruption, including Spatter, Speckle Noise, and Defocus Blur, showing the weak robustness of the agent. To improve the robustness of embodied navigation agents to various visual corruptions, we propose a navigation framework called Regularized Denoising Masked AutoEncoders Navigation (RDMAE-Nav). In a nutshell, RDMAE-Nav mainly consists of two modules: a visual module and a policy module. In the visual module, a self-supervised pretraining method, dubbed Regularized Denoising Masked AutoEncoders (RDMAE), is designed to enable the Vision Transformers (ViT)-based visual encoder to learn robust representations. The bidirectional Kullback–Leibler divergence is introduced in RDMAE as the regularization term for a denoising masked modeling task. Specifically, RDMAE mitigates the gap between clean and noisy image representations by minimizing the bidirectional Kullback–Leibler divergence. Then, the visual encoder is pretrained by RDMAE. In contrast to existing works, RDMAE-Nav applies denoising masked visual pretraining for PointGoal navigation to improve robustness to various visual corruptions. Finally, the pretrained visual encoder with frozen weights is applied to extract robust visual representations for policy learning in the RDMAE-Nav. Extensive experiments show that RDMAE-Nav performs competitively compared with state of the arts (SOTAs) on various visual corruptions. In detail, RDMAE-Nav performs the absolute improvement: 28.2% in SR and 23.68% in SPL under Spatter; 2.28% in SR and 6.41% in SPL under Speckle Noise; and 9.46% in SR and 9.55% in SPL under Defocus Blur.Keywords
Funding Information
- National Key Research and Development Program of China (2021YFB2501403, STS20201600200122)
- Science and Technology Service Network Initiative Program of the Chinese Academy of Sciences (2021YFB2501403, STS20201600200122)
This publication has 18 references indexed in Scilit:
- A Survey of Embodied AI: From Simulators to Research TasksIEEE Transactions on Emerging Topics in Computational Intelligence, 2022
- Self-Supervised Domain Adaptation for Visual Navigation with Global Map Consistency2022
- RobustNav: Towards Benchmarking Robustness in Embodied Navigation2021
- Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation2021
- Bi-Directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation AgentsIEEE Robotics and Automation Letters, 2021
- Embodied Visual Navigation With Automatic Curriculum Learning in Real EnvironmentsIEEE Robotics and Automation Letters, 2021
- RoboTHOR: An Open Simulation-to-Real Embodied AI Platform2020
- Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation2017
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Long Short-Term MemoryNeural Computation, 1997