Online approximate optimal control for affine non‐linear systems with unknown internal dynamics using adaptive dynamic programming
Open Access
- 1 November 2014
- journal article
- research article
- Published by Institution of Engineering and Technology (IET) in IET Control Theory & Applications
- Vol. 8 (16) , 1676-1688
Abstract
In this study, a novel online adaptive dynamic programming (ADP)‐based algorithm is developed for solving the optimal control problem of affine non‐linear continuous‐time systems with unknown internal dynamics. The present algorithm employs an observer–critic architecture to approximate the Hamilton–Jacobi–Bellman equation. Two neural networks (NNs) are used in this architecture: an NN state observer is constructed to estimate the unknown system dynamics and a critic NN is designed to derive the optimal control instead of typical action–critic dual networks employed in traditional ADP algorithms. Based on the developed architecture, the observer NN and the critic NN are tuned simultaneously. Meanwhile, unlike existing tuning laws for the critic, the newly developed critic update rule not only ensures convergence of the critic to the optimal control but also guarantees stability of the closed‐loop system. No initial stabilising control is required, and by using recorded and instantaneous data simultaneously for the adaptation of the critic, the restrictive persistence of excitation condition is relaxed. In addition, Lyapunov direct method is utilised to demonstrate the uniform ultimate boundedness of the weights of the observer NN and the critic NN. Finally, an example is provided to verify the effectiveness of the present approach.Keywords
Funding Information
- National Natural Science Foundation of China (61034002, 61233001, 61273140, 61304086, 61374105)
- Natural Science Foundation of Beijing Municipality (4132078)
This publication has 37 references indexed in Scilit:
- Neural‐network‐based online optimal control for uncertain non‐linear continuous‐time systems with control constraintsIET Control Theory & Applications, 2013
- Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraintsInternational Journal of Control, 2013
- Numerical adaptive learning control scheme for discrete‐time non‐linear systemsIET Control Theory & Applications, 2013
- Finite-Approximation-Error-Based Optimal Control Approach for Discrete-Time Nonlinear SystemsIEEE Transactions on Cybernetics, 2013
- An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputsInformation Sciences, 2013
- Optimal control for discrete-time affine non-linear systems using general value iterationIET Control Theory & Applications, 2012
- Optimal Control2012
- Approximate Dynamic Programming2011
- Adaptive Dynamic Programming: An IntroductionIEEE Computational Intelligence Magazine, 2009
- Adaptive dynamic programmingIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2002