Learning From Demonstrations Using Signal Temporal Logic in Stochastic and Continuous Domains
- 25 June 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (4) , 6250-6257
Abstract
Learning control policies that are safe, robust and interpretable are prominent challenges in developing robotic systems. Learning-from-demonstrations with formal logic is an arising paradigm in reinforcement learning to estimate rewards and extract robot control policies that seek to overcome these challenges. In this approach, we assume that mission-level specifications for the robotic system are expressed in a suitable temporal logic such as Signal Temporal Logic (STL). The main idea is to automatically infer rewards from user demonstrations (that could be suboptimal or incomplete) by evaluating and ranking them w.r.t. the given STL specifications. In contrast to existing work that focuses on deterministic environments and discrete state spaces, in this letter, we propose significant extensions that tackle stochastic environments and continuous state spaces.Keywords
Funding Information
- National Science Foundation (SHF-2048094, CCF-1837131, CNS-1932620)
- Toyota R&D
This publication has 15 references indexed in Scilit:
- Safety-Aware Apprenticeship Learning2018
- Behavioral Cloning from Observation2018
- Quantitative monitoring of STL with edit distanceFormal Methods in System Design, 2018
- Reinforcement learning with temporal logic rewards2017
- Learning from Demonstrations with High-Level Side Information2017
- Robust online monitoring of signal temporal logicFormal Methods in System Design, 2017
- Imitation LearningACM Computing Surveys, 2017
- Q-Learning for robust satisfaction of signal temporal logic specifications2016
- Robustness of temporal logic specifications for continuous-time signalsTheoretical Computer Science, 2009
- Apprenticeship learning via inverse reinforcement learning2004