State Constrained Stochastic Optimal Control for Continuous and Hybrid Dynamical Systems Using DFBSDE
Preprint
Abstract
We develop a computationally efficient learning-based forward-backward stochastic differential equations (FBSDE) controller for both continuous and hybrid dynamical (HD) systems subject to stochastic noise and state constraints. Solutions to stochastic optimal control (SOC) problems satisfy the Hamilton-Jacobi-Bellman (HJB) equation. Using current FBSDE-based solutions, the optimal control can be obtained from the HJB equations using deep neural networks (e.g., long short-term memory (LSTM) networks). To ensure the learned controller respects the constraint boundaries, we enforce the state constraints using a soft penalty function. In addition to previous works, we adapt the deep FBSDE (DFBSDE) control framework to handle HD systems consisting of continuous dynamics and a deterministic discrete state change. We demonstrate our proposed algorithm in simulation on a continuous nonlinear system (cart-pole) and a hybrid nonlinear system (five-link biped).Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Safe Navigation and Obstacle Avoidance Using Differentiable Optimization Based Control Barrier FunctionsIEEE Robotics and Automation Letters, 2023
- Navigating over 3D environments while minimizing cumulative exposure to hazardous fieldsAutomatica, 2020
- Whole-Body Nonlinear Model Predictive Control Through Contacts for QuadrupedsIEEE Robotics and Automation Letters, 2018
- Stochastic MPC with offline uncertainty samplingAutomatica, 2017
- An Introduction to Trajectory Optimization: How to Do Your Own Direct CollocationSIAM Review, 2017
- Models, feedback control, and open problems of 3D bipedal robotic walkingAutomatica, 2014
- Stochastic optimal control via Bellman's principleAutomatica, 2003
- RABBIT: a testbed for advanced control theoryIEEE Control Systems, 2003