Adaptive optimal controllers based on Generalized Policy Iteration in a continuous-time framework
- 1 June 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1402-1409
Abstract
In this paper we present two adaptive algorithms which offer solution to the continuous-time optimal control problem for nonlinear, affine in the inputs, time-invariant systems. Both algorithms were developed based on the generalized policy iteration technique and involve adaptation of two neural network structures namely actor, providing the control signal, and critic, performing evaluation of the control performance. Despite the similarities, the two adaptive algorithms differ in the manner in which the adaptation takes place, required knowledge on the system dynamics, and formulation of the persistence of excitation requirement. The main difference is that one algorithm uses sequential adaptation of the actor and critic structures, i.e. while one is trained the other one is kept constant, while for the second algorithm the two neural networks are trained synchronously in a continuous-time fashion. The two algorithms are described in detail and proof of convergence is provided. Simulation results of applying the two algorithms for finding the optimal state feedback controller of a nonlinear system are also presented.Keywords
This publication has 12 references indexed in Scilit:
- Online policy iteration based algorithms to solve the continuous-time infinite horizon optimal control problem2009
- Adaptive optimal control for continuous-time linear systems based on policy iterationAutomatica, 2009
- Adaptive optimal control algorithm for continuous-time nonlinear systems based on policy iteration2008
- Continuous-Time Adaptive CriticsIEEE Transactions on Neural Networks, 2007
- Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approachAutomatica, 2005
- Adaptive dynamic programmingIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2002
- Reinforcement Learning in Continuous Time and SpaceNeural Computation, 2000
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equationAutomatica, 1997
- Adaptive critic designsIEEE Transactions on Neural Networks, 1997