Online reinforcement learning for a class of partially unknown continuous‐time nonlinear systems via value iteration
Authors:
- 21 December 2017
- journal article
- research article
- Published by Wiley in Optimal Control Applications and Methods
- Vol. 39 (2) , 1011-1028
Abstract
Summary: In this paper, a modified value iteration–based approximate dynamic programming method is proposed for a class of affine nonlinear continuous‐time systems, whose dynamics are partially unknown. The value iteration algorithm is established in an online fashion, and the convergence proof is given. To attenuate the effect caused by the unascertained characteristics of the system dynamics, the integral reinforcement learning scheme is also used. In the proposed approximate dynamic programming method, it is emphasized that the single‐network structure is utilized to estimate the value functions and the control policies. That is, the iteration process is implemented on the actor/critic structure, in which case only the critic NN is required to be identified. Then, the least‐squares scheme is derived for the NN weights updating. Finally, a linear system and a nonlinear system are tested to evaluate the performance of the proposed online value iteration algorithm. Both of the examples show the feasibility and effectiveness of the proposed algorithms.Keywords
Funding Information
- National Natural Science Foundation of China (61428301, 61433004, 61627809, 61621004)
This publication has 57 references indexed in Scilit:
- Online adaptive algorithm for optimal control with integral reinforcement learningInternational Journal of Robust and Nonlinear Control, 2013
- An approximate dynamic programming method for multi‐input multi‐output nonlinear systemOptimal Control Applications and Methods, 2011
- Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problemAutomatica, 2010
- Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systemsNeural Networks, 2009
- Adaptive optimal control for continuous-time linear systems based on policy iterationAutomatica, 2009
- Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity controlAutomatica, 2007
- Adaptive Critic Designs for Discrete-Time Zero-Sum Games With Application to $H_{\infty}$ ControlIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2007
- Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approachAutomatica, 2005
- Adaptive dynamic programmingIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2002
- Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networksNeural Networks, 1990