Planning With Uncertain Specifications (PUnS)
- 28 February 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 5 (2) , 3414-3421
Abstract
Reward engineering is crucial to high performance in reinforcement learning systems. Prior research into reward design has largely focused on Markovian functions representing the reward. While there has been research into expressing non-Markovian rewards as linear temporal logic (LTL) formulas, this has been limited to a single formula serving as the task specification. However, in many real-world applications, task specifications can only be expressed as a belief over LTL formulas. In this paper, we introduce planning with uncertain specifications (PUnS), a novel formulation that addresses the challenge posed by non-Markovian specifications expressed as beliefs over LTL formulas. We present four criteria that capture the semantics of satisfying a belief over specifications for different applications, and analyze the qualitative implications of these criteria within a synthetic domain. We demonstrate the existence of an equivalent markov decision process (MDP) for any instance of PUnS. Finally, we demonstrate our approach on the real-world task of setting a dinner table automatically with a robot that inferred task specifications from human demonstrations.Keywords
All Related Versions
Funding Information
- ARL (ARL DCIST CRA W911NF-17-2-0181)
This publication has 20 references indexed in Scilit:
- Active Preference-Based Learning of Reward Functions2017
- Q-Learning for robust satisfaction of signal temporal logic specifications2016
- Human-level control through deep reinforcement learningNature, 2015
- Temporal-Logic-Based Reactive Mission and Motion PlanningIEEE Transactions on Robotics, 2009
- Deterministic planning in the fifth international planning competition: PDDL3 and experimental evaluation of the plannersArtificial Intelligence, 2009
- A heuristic search approach to planning with temporally extended preferencesArtificial Intelligence, 2009
- Using temporal logics to express search control knowledge for planningArtificial Intelligence, 2000
- Simple On-the-fly Automatic Verification of Linear Temporal Logic1996
- An automata-theoretic approach to linear temporal logic1996
- Q-learningMachine Learning, 1992