SEAL: SEmantic-Augmented Imitation Learning via Language Model
Preprint
Abstract
Hierarchical Imitation Learning (HIL) is a promising approach for tackling long-horizon decision-making tasks. While it is a challenging task due to the lack of detailed supervisory labels for sub-goal learning, and reliance on hundreds to thousands of expert demonstrations. In this work, we introduce SEAL, a novel framework that leverages Large Language Models (LLMs)'s powerful semantic and world knowledge for both specifying sub-goal space and pre-labeling states to semantically meaningful sub-goal representations without prior knowledge of task hierarchies. SEAL employs a dual-encoder structure, combining supervised LLM-guided sub-goal learning with unsupervised Vector Quantization (VQ) for more robust sub-goal representations. Additionally, SEAL incorporates a transition-augmented low-level planner for improved adaptation to sub-goal transitions. Our experiments demonstrate that SEAL outperforms state-of-the-art HIL methods and LLM-based planning approaches, particularly in settings with small expert datasets and complex long-horizon tasks.Keywords
All Related Versions
This publication has 4 references indexed in Scilit:
- What Matters in Language Conditioned Robotic Imitation Learning Over Unstructured DataIEEE Robotics and Automation Letters, 2022
- Computational Benefits of Intermediate Rewards for Goal-Reaching Policy LearningJournal of Artificial Intelligence Research, 2022
- Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring2021
- A Vector Quantized Variational Autoencoder (VQ-VAE) Autoregressive Neural $F_0$ Model for Statistical Parametric Speech SynthesisIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019