Information Geometry of Exponentiated Gradient: Convergence beyond L-Smoothness
Preprint
Abstract
We study the minimization of smooth, possibly nonconvex functions over the positive orthant, a key setting in Poisson inverse problems, using the exponentiated gradient (EG) method. Interpreting EG as Riemannian gradient descent (RGD) with the $e$-Exp map from information geometry as a retraction, we prove global convergence under weak assumptions -- without the need for $L$-smoothness -- and finite termination of Riemannian Armijo line search. Numerical experiments, including an accelerated variant, highlight EG's practical advantages, such as faster convergence compared to RGD based on interior-point geometry.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- An Introduction to Optimization on Smooth Manifolds2023
- Global convergence of Hager–Zhang type Riemannian conjugate gradient methodApplied Mathematics and Computation, 2022
- Sufficient Descent Riemannian Conjugate Gradient MethodsJournal of Optimization Theory and Applications, 2021
- Convergence of the Exponentiated Gradient Method with Armijo Line SearchJournal of Optimization Theory and Applications, 2018
- A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and ApplicationsMathematics of Operations Research, 2017
- The Information Geometry of Mirror DescentIEEE Transactions on Information Theory, 2015
- Optimization Algorithms on Matrix Manifolds2008
- Interior Gradient and Proximal Methods for Convex and Conic OptimizationSIAM Journal on Optimization, 2006
- Mirror descent and nonlinear projected subgradient methods for convex optimizationOperations Research Letters, 2003
- Exponentiated Gradient versus Gradient Descent for Linear PredictorsInformation and Computation, 1997