A Framework of Inertial Alternating Direction Method of Multipliers for Non-Convex Non-Smooth Optimization
Preprint
Abstract
In this paper, we propose an algorithmic framework, dubbed inertial alternating direction methods of multipliers (iADMM), for solving a class of nonconvex nonsmooth multiblock composite optimization problems with linear constraints. Our framework employs the general minimization-majorization (MM) principle to update each block of variables so as to not only unify the convergence analysis of previous ADMM that use specific surrogate functions in the MM step, but also lead to new efficient ADMM schemes. To the best of our knowledge, in the nonconvex nonsmooth setting, ADMM used in combination with the MM principle to update each block of variables, and ADMM combined with \emph{inertial terms for the primal variables} have not been studied in the literature. Under standard assumptions, we prove the subsequential convergence and global convergence for the generated sequence of iterates. We illustrate the effectiveness of iADMM on a class of nonconvex low-rank representation problems.Keywords
All Related Versions
This publication has 46 references indexed in Scilit:
- A Globally Convergent Algorithm for Nonconvex Optimization Based on Block Coordinate UpdateJournal of Scientific Computing, 2017
- A block coordinate variable metric forward–backward algorithmJournal of Global Optimization, 2016
- On the Global and Linear Convergence of the Generalized Alternating Direction Method of MultipliersJournal of Scientific Computing, 2015
- Proximal alternating linearized minimization for nonconvex and nonsmooth problemsMathematical Programming, 2013
- A Splitting Method for Orthogonality Constrained ProblemsJournal of Scientific Computing, 2013
- A feasible method for optimization with orthogonality constraintsMathematical Programming, 2012
- Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methodsMathematical Programming, 2011
- A coordinate gradient descent method for nonsmooth separable minimizationMathematical Programming, 2007
- On the convergence of the proximal algorithm for nonsmooth functions involving analytic featuresMathematical Programming, 2007
- On search directions for minimization algorithmsMathematical Programming, 1973