Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions
Authors:
- Desong DuDDDesong Du
- Shaohang HanSHShaohang Han
- Naiming QiNQNaiming Qi
- Haitham Bou AmmarHAHaitham Bou Ammar
- Jun WangJWJun Wang
- Wei PanWPWei Pan
Preprint
Abstract
Reinforcement learning (RL) exhibits impressive performance when managing complicated control tasks for robots. However, its wide application to physical robots is limited by the absence of strong safety guarantees. To overcome this challenge, this paper explores the control Lyapunov barrier function (CLBF) to analyze the safety and reachability solely based on data without explicitly employing a dynamic model. We also proposed the Lyapunov barrier actor-critic (LBAC), a model-free RL algorithm, to search for a controller that satisfies the data-based approximation of the safety and reachability conditions. The proposed approach is demonstrated through simulation and real-world robot control experiments, i.e., a 2D quadrotor navigation task. The experimental findings reveal this approach's effectiveness in reachability and safety, surpassing other model-free RL methods.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Smooth converse Lyapunov-barrier theorems for asymptotic stability with safety constraints and reach-avoid-stay specificationsAutomatica, 2022
- Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement LearningAnnual Review of Control, Robotics, and Autonomous Systems, 2022
- Reinforcement learning control of constrained dynamic systems with uniformly ultimate boundedness stability guaranteeAutomatica, 2021
- Recovery RL: Safe Reinforcement Learning With Learned Recovery ZonesIEEE Robotics and Automation Letters, 2021
- Model-Based Meta-Reinforcement Learning for Flight With Suspended PayloadsIEEE Robotics and Automation Letters, 2021
- Actor-Critic Reinforcement Learning for Control With Stability GuaranteeIEEE Robotics and Automation Letters, 2020
- Learning-Based Control: A Tutorial and Some Recent Results2020
- Low-Level Control of a Quadrotor With Deep Model-Based Reinforcement LearningIEEE Robotics and Automation Letters, 2019
- Barrier-Certified Adaptive Reinforcement Learning With Applications to Brushbot NavigationIEEE Transactions on Robotics, 2019
- Stabilization with guaranteed safety using Control Lyapunov–Barrier FunctionAutomatica, 2016