Unifying mirror descent and dual averaging
Authors:
- Anatoli JuditskyAJAnatoli Juditsky
- Joon KwonJKJoon Kwon
- Éric MoulinesÉMÉric Moulines
Preprint
Abstract
We introduce and analyze a new family of first-order optimization algorithms which generalizes and unifies both mirror descent and dual averaging. Within the framework of this family, we define new algorithms for constrained optimization that combines the advantages of mirror descent and dual averaging. Our preliminary simulation study shows that these new algorithms significantly outperform available methods in some situations.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Algorithms of Inertial Mirror Descent in Convex Problems of Stochastic OptimizationAutomation and Remote Control, 2018
- Dual subgradient algorithms for large-scale nonsmooth learning problemsMathematical Programming, 2013
- Dual Averaging for Distributed Optimization: Convergence Analysis and Network ScalingIEEE Transactions on Automatic Control, 2011
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse ProblemsSIAM Journal on Imaging Sciences, 2009
- Robust Stochastic Approximation Approach to Stochastic ProgrammingSIAM Journal on Optimization, 2009
- Primal-dual subgradient methods for convex problemsMathematical Programming, 2007
- Dual extrapolation and its applications to solving variational inequalities and related problemsMathematical Programming, 2006
- Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point ProblemsSIAM Journal on Optimization, 2004
- Mirror descent and nonlinear projected subgradient methods for convex optimizationOperations Research Letters, 2003
- Convergence Analysis of a Proximal-Like Minimization Algorithm Using Bregman FunctionsSIAM Journal on Optimization, 1993