Theoretical and Numerical Analysis of Approximate Dynamic Programming with Approximation Errors
Author:
- 1 February 2016
- journal article
- research article
- Published by American Institute of Aeronautics and Astronautics (AIAA) in Journal of Guidance, Control, and Dynamics
- Vol. 39 (2) , 301-311
Abstract
This study is aimed at answering the question of how the approximation errors at each iteration of approximate dynamic programming affect the quality of the final results, considering the fact that errors at each iteration affect the next iteration. To this goal, convergence of value iteration scheme of approximate dynamic programming for deterministic nonlinear optimal control problems with discrete-time known dynamics subject to an undiscounted known cost functions is investigated while considering the errors existing in approximating respective functions. The boundedness of the results around the optimal solution is obtained based on quantities that are known in a general optimal control problem and assumptions that are verifiable. Moreover, because the presence of the approximation errors leads to the deviation of the results from optimality, sufficient conditions for stability of the system operated by the result obtained after a finite number of value iterations, along with an estimation of its region of attraction, are derived in terms of a calculable upper bound of the control approximation error. Finally, the process of implementation of the method on an orbital maneuver problem is investigated, through which the assumptions made in the theoretical developments are verified and the sufficient conditions are applied for guaranteeing stability and near optimality.Keywords
All Related Versions
This publication has 32 references indexed in Scilit:
- Optimal control of uncertain quantized linear discrete‐time systemsInternational Journal of Adaptive Control and Signal Processing, 2014
- Fixed-final-time optimal control of nonlinear systems with terminal constraintsNeural Networks, 2013
- Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programmingAutomatica, 2012
- A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration AlgorithmIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008
- Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence ProofIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008
- Helicopter trimming and tracking control using direct neural dynamic programmingIEEE Transactions on Neural Networks, 2003
- Comparison of heuristic dynamic programming and dual heuristic programming adaptive critics for neurocontrol of a turbogeneratorIEEE Transactions on Neural Networks, 2002
- Adaptive Critic-Based Neural Networks for Agile Missile ControlJournal of Guidance, Control, and Dynamics, 2002
- Adaptive critic designsIEEE Transactions on Neural Networks, 1997
- Adaptive-critic-based neural networks for aircraft optimal controlJournal of Guidance, Control, and Dynamics, 1996