Probabilistically Guaranteed Satisfaction of Temporal Logic Constraints During Reinforcement Learning
Authors:
- Derya AksarayDADerya Aksaray
- Yasin YaziciogluYYYasin Yazicioglu
- Ahmet Semi AsarkayaAAAhmet Semi Asarkaya
Preprint
Abstract
We propose a novel constrained reinforcement learning method for finding optimal policies in Markov Decision Processes while satisfying temporal logic constraints with a desired probability throughout the learning process. An automata-theoretic approach is proposed to ensure the probabilistic satisfaction of the constraint in each episode, which is different from penalizing violations to achieve constraint satisfaction after a sufficiently large number of episodes. The proposed approach is based on computing a lower bound on the probability of constraint satisfaction and adjusting the exploration behavior as needed. We present theoretical results on the probabilistic constraint satisfaction achieved by the proposed approach. We also numerically demonstrate the proposed idea in a drone scenario, where the constraint is to perform periodically arriving pick-up and delivery tasks and the objective is to fly over high-reward zones to simultaneously perform aerial monitoring.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Absorption in Time-Varying Markov Chains: Graph-Based ConditionsIEEE Control Systems Letters, 2020
- STyLuS*: A Temporal Logic Optimal Control Synthesis Algorithm for Large-Scale Multi-Robot SystemsThe International Journal of Robotics Research, 2020
- Distributed Path Planning for Executing Cooperative Tasks with Time WindowsIFAC-PapersOnLine, 2019
- A General Safety Framework for Learning-Based Control in Uncertain Robotic SystemsIEEE Transactions on Automatic Control, 2018
- Time window temporal logicTheoretical Computer Science, 2017
- Communication-Free Distributed Coverage for Networked SystemsIEEE Transactions on Control of Network Systems, 2016
- Receding horizon temporal logic control in dynamic environmentsThe International Journal of Robotics Research, 2014
- Temporal-Logic-Based Reactive Mission and Motion PlanningIEEE Transactions on Robotics, 2009
- Integrating Guidance into Relational Reinforcement LearningMachine Learning, 2004