Sablas: Learning Safe Control for Black-Box Dynamical Systems
Open Access
- 13 January 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 1928-1935
Abstract
Control certificates based on barrier functions have been a powerful tool to generate probably safe control policies for dynamical systems. However, existing methods based on barrier certificates are normally for white-box systems with differentiable dynamics, which makes them inapplicable to many practical applications where the system is a black-box and cannot be accurately modeled. On the other side, model-free reinforcement learning (RL) methods for black-box systems suffer from lack of safety guarantees and low sampling efficiency. In this paper, we propose a novel method that can learn safe control policies and barrier certificates for black-box dynamical systems, without requiring for an accurate system model. Our method re-designs the loss function to back-propagate gradient to the control policy even when the black-box dynamical system is non-differentiable, and we show that the safety certificates hold on the black-box system. Empirical results in simulation show that our method can significantly improve the performance of the learned policies by achieving nearly 100% safety and goal reaching rates using much fewer training samples, compared to state-of-the-art black-box safe control methods. Our learned agents can also generalize to unseen scenarios while keeping the original performance. The source code can be found at https://github.com/Zengyi-Qin/bcbf .Keywords
All Related Versions
Funding Information
- Defense Science and Technology Agency in Singapore
This publication has 28 references indexed in Scilit:
- Control Contraction Metrics: Convex and Intrinsic Criteria for Nonlinear Feedback DesignIEEE Transactions on Automatic Control, 2017
- Control Barrier Function Based Quadratic Programs for Safety Critical SystemsIEEE Transactions on Automatic Control, 2016
- Control Barrier Certificates for Safe Swarm BehaviorIFAC-PapersOnLine, 2015
- Practical relative degree in black-box control2012
- Local stability analysis using simulations and sum-of-squares programmingAutomatica, 2008
- A tutorial on sum of squares techniques for systems analysis2005
- A survey on Nonlinear Ship Control: from Theory to PracticeIFAC Proceedings Volumes, 2000
- On Contraction Analysis for Non-linear SystemsAutomatica, 1998
- Robust Nonlinear Control Design1996
- Nonlinear Control Systems1995