Learning to Reason and Navigate: Parameter Efficient Action Planning with Large Language Models
Authors:
Preprint
Abstract
The remote embodied referring expression (REVERIE) task requires an agent to navigate through complex indoor environments and localize a remote object specified by high-level instructions, such as "bring me a spoon", without pre-exploration. Hence, an efficient navigation plan is essential for the final success. This paper proposes a novel parameter-efficient action planner using large language models (PEAP-LLM) to generate a single-step instruction at each location. The proposed model consists of two modules, LLM goal planner (LGP) and LoRA action planner (LAP). Initially, LGP extracts the goal-oriented plan from REVERIE instructions, including the target object and room. Then, LAP generates a single-step instruction with the goal-oriented plan, high-level instruction, and current visual observation as input. PEAP-LLM enables the embodied agent to interact with LAP as the path planner on the fly. A simple direct application of LLMs hardly achieves good performance. Also, existing hard-prompt-based methods are error-prone in complicated scenarios and need human intervention. To address these issues and prevent the LLM from generating hallucinations and biased information, we propose a novel two-stage method for fine-tuning the LLM, consisting of supervised fine-tuning (STF) and direct preference optimization (DPO). SFT improves the quality of generated instructions, while DPO utilizes environmental feedback. Experimental results show the superiority of our proposed model on REVERIE compared to the previous state-of-the-art.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- Learning from Unlabeled 3D Environments for Vision-and-Language Navigation2022
- Scene-Intuitive Agent for Remote Embodied Visual Grounding2021
- SOON: Scenario Oriented Object Navigation with Graph-based Exploration2021
- VLN↻BERT: A Recurrent Vision-and-Language BERT for Navigation2021
- Room-and-Object Aware Knowledge Reasoning for Remote Embodied Referring Expression2021
- Structured Scene Memory for Vision-Language Navigation2021
- Gold Seeker: Information Gain From Policy Distributions for Goal-Oriented Vision-and-Langauge Reasoning2020
- REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments2020
- What's to Know? Uncertainty as a Guide to Asking Goal-Oriented Questions2019
- TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments2019