Error Bounds of Adaptive Dynamic Programming Algorithms for Solving Undiscounted Optimal Control Problems
Authors:
- 3 March 2015
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 26 (6) , 1323-1334
Abstract
In this paper, we establish error bounds of adaptive dynamic programming algorithms for solving undiscounted infinite-horizon optimal control problems of discrete-time deterministic nonlinear systems. We consider approximation errors in the update equations of both value function and control policy. We utilize a new assumption instead of the contraction assumption in discounted optimal control problems. We establish the error bounds for approximate value iteration based on a new error condition. Furthermore, we also establish the error bounds for approximate policy iteration and approximate optimistic policy iteration algorithms. It is shown that the iterative approximate value function can converge to a finite neighborhood of the optimal value function under some conditions. To implement the developed algorithms, critic and action neural networks are used to approximate the value function and control policy, respectively. Finally, a simulation example is given to demonstrate the effectiveness of the developed algorithms.Keywords
Funding Information
- National Natural Science Foundation of China (61034002, 61233001, 61273140, 61304086, 61374105)
- Beijing Natural Science Foundation (4132078)
This publication has 46 references indexed in Scilit:
- Optimal control for discrete-time affine non-linear systems using general value iterationIET Control Theory & Applications, 2012
- Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programmingAutomatica, 2012
- Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approachNeurocomputing, 2012
- A self-learning scheme for residential energy system control and managementNeural Computing and Applications, 2011
- Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With $\varepsilon$-Error BoundIEEE Transactions on Neural Networks, 2010
- Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergenceNeural Networks, 2009
- Performance Bounds in $L_p$‐norm for Approximate Value IterationSIAM Journal on Control and Optimization, 2007
- A Self-Learning Call Admission Control Scheme for CDMA Cellular NetworksIEEE Transactions on Neural Networks, 2005
- Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approachAutomatica, 2005
- Construction of Suboptimal Control SequencesSIAM Journal on Control, 1967