Performance of various BFGS implementations with limited precision second-order information
Authors:
- 1 April 2004
- journal article
- research article
- Published by Cambridge University Press (CUP) in The ANZIAM Journal
- Vol. 45 (4) , 511-522
Abstract
The BFGS formula is arguably the most well known and widely used update method for quasi-Newton algorithms. Some authors have claimed that updating approximate Hessian information via the BFGS formula with a Cholesky factorisation offers greater numerical stability than the more straightforward approach of performing the update directly. Other authors have claimed that no such advantage exists and that any such improvement is probably due to early implementations of the DFP formula in conjunction with low accuracy line searches. This paper supports the claim that there is no discernible advantage in choosing factorised implementations (over non-factorised implementations) of BFGS methods when approximate Hessian information is available to full machine precision. However the results presented in this paper show that a factorisation strategy has clear advantages when approximate Hessian information is available only to limited precision. These results show that a conjugate directions factorisation outperforms the other methods considered in this paper (including Cholesky factorisation).Keywords
This publication has 9 references indexed in Scilit:
- Conjugate Grids for Unconstrained OptimisationComputational Optimization and Applications, 2004
- Performance of various BFGS implementations with limited precision second-order informationThe ANZIAM Journal, 2004
- A Revised Modified Cholesky Factorization AlgorithmSIAM Journal on Optimization, 1999
- A conjugate direction implementation of the BFGS algorithm with automatic scalingThe Journal of the Australian Mathematical Society. Series B. Applied Mathematics, 1989
- Updating conjugate directions by the BFGS formulaMathematical Programming, 1987
- Testing Unconstrained Optimization SoftwareACM Transactions on Mathematical Software, 1981
- Optimally conditioned optimization algorithms without line searchesMathematical Programming, 1975
- Rank-one and Rank-two Corrections to Positive Definite Matrices Expressed in Product FormIMA Journal of Applied Mathematics, 1973
- Quasi-Newton Methods for Unconstrained OptimizationIMA Journal of Applied Mathematics, 1972