Fast Decentralized Nonconvex Finite-Sum Optimization with Recursive Variance Reduction
Authors:
- Ran XinRXRan Xin
- Usman A. KhanUKUsman A. Khan
- Soummya KarSKSoummya Kar
- 5 January 2022
- journal article
- research article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Optimization
- Vol. 32 (1) , 1-28
Abstract
This paper considers decentralized minimization of $N:=nm$ smooth nonconvex cost functions equally divided over a directed network of $n$ nodes. Specifically, we describe a stochastic first-order g...
Keywords
All Related Versions
Funding Information
- National Science Foundation (1513936, 1903972, 1935555)
This publication has 38 references indexed in Scilit:
- NEXT: In-Network Nonconvex OptimizationIEEE Transactions on Signal and Information Processing over Networks, 2016
- On Convergence Rate of Distributed Stochastic Gradient Algorithm for Convex Optimization with Inequality ConstraintsSIAM Journal on Control and Optimization, 2016
- On the Learning Behavior of Adaptive Networks—Part I: Transient AnalysisIEEE Transactions on Information Theory, 2015
- EXTRA: An Exact First-Order Algorithm for Decentralized Consensus OptimizationSIAM Journal on Optimization, 2015
- A Proximal Stochastic Gradient Method with Progressive Variance ReductionSIAM Journal on Optimization, 2014
- Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic ProgrammingSIAM Journal on Optimization, 2013
- Distributed Strategies for Generating Weight-Balanced and Doubly Stochastic DigraphsEuropean Journal of Control, 2012
- Distributed Stochastic Subgradient Projection Algorithms for Convex OptimizationJournal of Optimization Theory and Applications, 2010
- Penalized likelihood regression for generalized linear models with non-quadratic penaltiesAnnals of the Institute of Statistical Mathematics, 2009
- Robust Stochastic Approximation Approach to Stochastic ProgrammingSIAM Journal on Optimization, 2009