Continuous-Time Stochastic Policy Iteration of Adaptive Dynamic Programming
- 30 June 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 53 (10) , 6375-6387
Abstract
In this article, we study the optimal control problem of continuous-time (CT) time-invariant nonlinear systems with stochastic nonlinear disturbances. A new stochastic adaptive dynamic programming (ADP) method is developed to solve the Hamilton–Jacobi–Bellman equation (HJBE). Under the conditional expectation, the value function and the control law are successively approximated simultaneously. The asymptotic stability of the closed-loop stochastic system in probability is analyzed by the stochastic Lyapunov direct method, and the convergence of the developed ADP method is given. Finally, four simulations illustrate the effectiveness of the developed method.Keywords
Funding Information
- National Key Research and Development Program of China (2021YFE0206100)
- science and technology development fund of the Macao special administrative region (FDCT-22-009-MISE)
- National Natural Science Foundation of China (62073321, U22A2046, 52172322)
- State Key Laboratory of Rail Traffic Control and Safety, Beijing Jiaotong University (RCS2021K005)
This publication has 44 references indexed in Scilit:
- Data-based approximate policy iteration for affine nonlinear continuous-time optimal control designAutomatica, 2014
- Adaptive dynamic programming as a theory of sensorimotor controlBiological Cybernetics, 2014
- Probabilistic DHP adaptive critic for nonlinear stochastic control systemsNeural Networks, 2013
- Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamicsNeural Computing and Applications, 2012
- Adaptive critic methods for stochastic systems with input-dependent noiseAutomatica, 2007
- Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approachAutomatica, 2005
- Uplink Power Adjustment in Wireless Communication Systems: A Stochastic Control AnalysisIEEE Transactions on Automatic Control, 2004
- Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equationAutomatica, 1997
- On a Matrix Riccati Equation of Stochastic ControlSIAM Journal on Control, 1968
- Dynamic ProgrammingScience, 1966