Physics-Model-Regulated Deep Reinforcement Learning Towards Safety & Stability Guarantees
- 13 December 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 17, 8306-8311
Abstract
Deep reinforcement learning (DRL) has demonstrated impressive success in solving complex control tasks by synthesizing control policies from data. However, the safety and stability of applying DRL to safety-critical systems remain a primary concern and challenging problem. To address the problem, we propose the Phy-DRL: a novel physics-model-regulated deep reinforcement learning framework. The Phy-DRL is novel in two architectural designs: a physics-model-regulated reward and residual control, which integrates physics-model-based control and data-driven control. The concurrent designs enable the Phy-DRL the mathematically provable safety and stability guarantees. Finally, the effectiveness of the Phy-DRL is validated by an inverted pendulum system. Additionally, the experimental results demonstrate that the Phy-DRL features remarkably accelerated training and enlarged reward.Keywords
Funding Information
- NSF (CPS-2311084)
This publication has 11 references indexed in Scilit:
- Bayesian controller fusion: Leveraging control priors in deep reinforcement learning for roboticsThe International Journal of Robotics Research, 2023
- Sℒ1-Simplex: Safe Velocity Regulation of Self-Driving Vehicles in Dynamic and Unforeseen EnvironmentsACM Transactions on Cyber-Physical Systems, 2023
- Quantifying Safety of Learning-based Self-Driving Control Using Almost-Barrier Functions2022
- RMA: Rapid Motor Adaptation for Legged Robots2021
- Stabilizing Neural Control Using Self-Learned Almost Lyapunov Critics2021
- Residual Reinforcement Learning for Robot Control2019
- A General Safety Framework for Learning-Based Control in Uncertain Robotic SystemsIEEE Transactions on Automatic Control, 2018
- Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks2017
- Linear Matrix Inequalities in System and Control Theory1994
- Nonlinear Systems1992