Novel Iteratively Preconditioned Gradient-Descent Algorithm via Successive Over-Relaxation Formulation
- 23 December 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems Letters
- Vol. 8, 3105-3110
Abstract
We devise a novel quasi-Newton algorithm for solving unconstrained convex optimization problems. The proposed algorithm is built on our previous framework of the iteratively preconditioned gradient-descent (IPG) algorithm. IPG utilized Richardson iteration to update a preconditioner matrix that approximates the inverse of the Hessian matrix. In this work, we substitute the Richardson iteration with a successive over-relaxation (SOR) formulation. The convergence guarantee of the proposed algorithm and its theoretical improvement over vanilla IPG are presented. The algorithm is used in a mobile robot position estimation problem for numerical validation using a moving horizon estimation (MHE) formulation. Compared with IPG, the results demonstrate an improved performance of the proposed algorithm in terms of computational time and the number of iterations needed for convergence, matching our theoretical results.Keywords
Funding Information
- DEVCOM Army Research Laboratory (W911NF-22-2-0040)
This publication has 21 references indexed in Scilit:
- Accelerating the Iteratively Preconditioned Gradient-Descent Algorithm using Momentum2023
- A novel flow update policy in solving traffic assignment problems: Successive over relaxation iteration methodTransportation Research Part E: Logistics and Transportation Review, 2023
- Efficient Successive Over Relaxation Detectors for Massive MIMOIEEE Transactions on Circuits and Systems I: Regular Papers, 2020
- Successive Over-Relaxation ${Q}$ -LearningIEEE Control Systems Letters, 2019
- Iterative Solution of Large Sparse Systems of Equations2016
- Successive Overrelaxation for Laplacian Support Vector MachineIEEE Transactions on Neural Networks and Learning Systems, 2014
- Symmetric SOR method for augmented systemsApplied Mathematics and Computation, 2006
- On generalized successive overrelaxation methods for augmented linear systemsNumerische Mathematik, 2005
- Constrained state estimation for nonlinear discrete-time systems: stability and moving horizon approximationsIEEE Transactions on Automatic Control, 2003
- Successive overrelaxation for support vector machinesIEEE Transactions on Neural Networks, 1999