Adjoint-based exact Hessian computation
Open Access
- 17 February 2021
- journal article
- research article
- Published by Springer Nature in BIT Numerical Mathematics
- Vol. 61 (2) , 503-522
Abstract
We consider a scalar function depending on a numerical solution of an initial value problem, and its second-derivative (Hessian) matrix for the initial value. The need to extract the information of the Hessian or to solve a linear system having the Hessian as a coefficient matrix arises in many research fields such as optimization, Bayesian estimation, and uncertainty quantification. From the perspective of memory efficiency, these tasks often employ a Krylov subspace method that does not need to hold the Hessian matrix explicitly and only requires computing the multiplication of the Hessian and a given vector. One of the ways to obtain an approximation of such Hessian-vector multiplication is to integrate the so-called second-order adjoint system numerically. However, the error in the approximation could be significant even if the numerical integration to the second-order adjoint system is sufficiently accurate. This paper presents a novel algorithm that computes the intended Hessian-vector multiplication exactly and efficiently. For this aim, we give a new concise derivation of the second-order adjoint system and show that the intended multiplication can be computed exactly by applying a particular numerical method to the second-order adjoint system. In the discussion, symplectic partitioned Runge–Kutta methods play an essential role.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Symplectic Runge--Kutta Schemes for Adjoint Equations, Automatic Differentiation, Optimal Control, and MoreSIAM Review, 2016
- Acoustic Inversion in Optoacoustic Tomography: A ReviewCurrent Medical Imaging, 2014
- The adjoint method in seismologyPhysics of the Earth and Planetary Interiors, 2006
- Regularized Newton Methods for Convex Minimization Problems with Singular SolutionsComputational Optimization and Applications, 2004
- Finite Difference of Adjoint or Adjoint of Finite Difference?Monthly Weather Review, 1997
- On Order Conditions for Partitioned Symplectic MethodsSIAM Journal on Numerical Analysis, 1997
- Partitioned Runge-Kutta Methods for Separable Hamiltonian ProblemsMathematics of Computation, 1993
- The second order adjoint analysis: Theory and applicationsMeteorology and Atmospheric Physics, 1992
- The role of the Hessian matrix in fitting models to measurementsJournal of Geophysical Research: Oceans, 1989
- Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspectsTellus, 1986