Programmatic Imitation Learning From Unlabeled and Noisy Demonstrations
- 5 April 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (6) , 4894-4901
Abstract
Imitation Learning (IL) is a promising paradigm for teaching robots to perform novel tasks using demonstrations. Most existing approaches for IL utilize neural networks (NN), however, these methods suffer from several well-known limitations: they 1) require large amounts of training data, 2) are hard to interpret, and 3) are hard to refine and adapt. There is an emerging interest in Programmatic Imitation Learning (PIL), which offers significant promise in addressing the above limitations. In PIL, the learned policy is represented in a programming language, making it amenable to interpretation and adaptation to novel settings. However, state-of-the-art PIL algorithms assume access to action labels and struggle to learn from noisy real-world demonstrations. In this paper, we propose PLUNDER, a novel PIL algorithm that addresses these shortcomings by synthesizing probabilistic programmatic policies that are particularly well-suited for modeling the uncertainties inherent in real-world demonstrations. Our approach leverages an EM loop to simultaneously infer the missing action labels and the most likely probabilistic policy. We benchmark PLUNDER against several established IL techniques, and demonstrate its superiority across five challenging imitation learning tasks under noise. PLUNDER policies outperform the next-best baseline by 19% and 17% in matching the given demonstrations and successfully completing the tasks, respectively.Keywords
All Related Versions
Funding Information
- National Science Foundation (CAREER-2046955, CCF-2319471, CCF-1762299, CCF-1918889, CNS-1908304, CCF-1901376, CNS-2120696, CCF-2210831)
This publication has 25 references indexed in Scilit:
- Programming-by-Demonstration for Long-Horizon Robot TasksProceedings of the ACM on Programming Languages, 2024
- STEADY: Simultaneous State Estimation and Dynamics Learning from Indirect Observations2022
- Neurosymbolic ProgrammingFoundations and Trends® in Programming Languages, 2021
- Learning robust manipulation tasks involving contact using trajectory parameterized probabilistic principal component analysis2020
- Learning to Drive by Imitation: An Overview of Deep Behavior Cloning MethodsIEEE Transactions on Intelligent Vehicles, 2020
- Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience2019
- The limits and potentials of deep learning for roboticsThe International Journal of Robotics Research, 2018
- Using probabilistic movement primitives in roboticsAutonomous Robots, 2017
- Program SynthesisFoundations and Trends® in Programming Languages, 2017
- Learning Transferable Policies for Monocular Reactive MAV Control2017