Sparse Learning-Based Approximate Dynamic Programming With Barrier Constraints
- 3 March 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems Letters
- Vol. 4 (3) , 743-748
Abstract
This letter provides an approximate online adaptive solution to the infinite-horizon optimal control problem for control-affine continuous-time nonlinear systems while formalizing system safety using barrier certificates. The use of a barrier function transform provides safety certificates to formalize system behavior. Specifically, using a barrier function, the system is transformed to aid in developing a controller which maintains the system in a pre-defined constrained region. To aid in online learning of the value function, the state-space is segmented into a number of user-defined segments. Off-policy trajectories are selected in each segment, and sparse Bellman error extrapolation is performed within each respective segment to generate an optimal policy within each segment. A Lyapunov-like stability analysis is included which proves uniformly ultimately bounded regulation in the presence of the barrier function transform and discontinuities. Simulation results are provided for a two-state dynamical system to compare the performance of the developed method to existing methods.Keywords
Funding Information
- Office of Naval Research (N00014-13-1-0151)
- Naval Engineering Education Consortium (N00174-18-1-0003)
- Air Force Office of Scientific Research (FA9550-18-1-0109, FA9550-19-1-0169)
- National Science Foundation (1762829)
This publication has 30 references indexed in Scilit:
- Approximate optimal trajectory tracking for continuous-time nonlinear systemsAutomatica, 2015
- Approximate $N$ -Player Nonzero-Sum Game Solution for an Uncertain Continuous Nonlinear SystemIEEE Transactions on Neural Networks and Learning Systems, 2014
- Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learningAutomatica, 2014
- Reinforcement -learning for optimal tracking control of linear discrete-time systems with unknown dynamicsAutomatica, 2014
- Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programmingInternational Journal of Control, 2013
- A novel actor–critic–identifier architecture for approximate optimal control of uncertain nonlinear systemsAutomatica, 2013
- Concurrent learning adaptive control of linear systems with exponentially convergent boundsInternational Journal of Adaptive Control and Signal Processing, 2012
- Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problemAutomatica, 2010
- Safety Verification of Hybrid Systems Using Barrier Certificates2004
- A calculus for computing Filippov's differential inclusion with application to the variable structure control of robot manipulatorsIEEE Transactions on Circuits and Systems, 1987