Linearly Solvable Stochastic Control Lyapunov Functions
Preprint
Abstract
This paper presents a new method for synthesizing stochastic control Lyapunov functions for a class of nonlinear stochastic control systems. The technique relies on a transformation of the classical nonlinear Hamilton-Jacobi-Bellman partial differential equation to a linear partial differential equation for a class of problems with a particular constraint on the stochastic forcing. This linear partial differential equation can then be relaxed to a linear differential inclusion, allowing for relaxed solutions to be generated using sum of squares programming. The resulting relaxed solutions are in fact viscosity super/subsolutions, and by the maximum principle are pointwise upper and lower bounds to the underlying value function, even for coarse polynomial approximations. Furthermore, the pointwise upper bound is shown to be a stochastic control Lyapunov function, yielding a method for generating nonlinear controllers with pointwise bounded distance from the optimal cost when using the optimal controller. These approximate solutions may be computed with non-increasing error via a hierarchy of semidefinite optimization problems. Finally, this paper develops a-priori bounds on trajectory suboptimality when using these approximate value functions, as well as demonstrates that these methods, and bounds, can be applied to a more general class of nonlinear systems not obeying the constraint on stochastic forcing. Simulated examples illustrate the methodology.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Semidefinite relaxations for stochastic optimal control policies2014
- Reinforcement Learning With Sequences of Motion Primitives for Robust ManipulationIEEE Transactions on Robotics, 2012
- Efficient computation of optimal actionsProceedings of the National Academy of Sciences of the United States of America, 2009
- Controlled Markov Processes and Viscosity Solutions2006
- Linear Theory for Control of Nonlinear Stochastic SystemsPhysical Review Letters, 2005
- Path integrals and symmetry breaking for optimal control theoryJournal of Statistical Mechanics: Theory and Experiment, 2005
- Relative entropy and efficiency measure for diffusion-mediated transport processesJournal of Physics A: General Physics, 2005
- Semidefinite programming relaxations for semialgebraic problemsMathematical Programming, 2003
- Stochastic nonlinear stabilization — I: A backstepping designSystems & Control Letters, 1997
- User’s guide to viscosity solutions of second order partial differential equationsBulletin of the American Mathematical Society, 1992