An Adaptive Multiparameter Penalty Selection Method for Multiconstraint and Multiblock ADMM
Authors:
- Luke LozenskiLLLuke Lozenski
- Michael T. McCannMMMichael T. McCann
- Brendt WohlbergBWBrendt Wohlberg
Preprint
Abstract
This work presents a new method for online selection of multiple penalty parameters for the alternating direction method of multipliers (ADMM) algorithm applied to optimization problems with multiple constraints or functionals with block matrix components. ADMM is widely used for solving constrained optimization problems in a variety of fields, including signal and image processing. Implementations of ADMM often utilize a single hyperparameter, referred to as the penalty parameter, which needs to be tuned to control the rate of convergence. However, in problems with multiple constraints, ADMM may demonstrate slow convergence regardless of penalty parameter selection due to scale differences between constraints. Accounting for scale differences between constraints to improve convergence in these cases requires introducing a penalty parameter for each constraint. The proposed method is able to adaptively account for differences in scale between constraints, providing robustness with respect to problem transformations and initial selection of penalty parameters. It is also simple to understand and implement. Our numerical experiments demonstrate that the proposed method performs favorably compared to a variety of existing penalty parameter selection methods.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- Non-stationary Douglas–Rachford and alternating direction method of multipliers: adaptive step-sizes and convergenceComputational Optimization and Applications, 2019
- ADMM for monotone operators: convergence analysis and ratesAdvances in Computational Mathematics, 2018
- Global Convergence of ADMM in Nonconvex Nonsmooth OptimizationJournal of Scientific Computing, 2018
- Parallel Multi-Block ADMM with o(1 / k) ConvergenceJournal of Scientific Computing, 2016
- Self Equivalence of the Alternating Direction Method of Multipliers2016
- On the Sublinear Convergence Rate of Multi-block ADMMJournal of the Operations Research Society of China, 2015
- On the Global and Linear Convergence of the Generalized Alternating Direction Method of MultipliersJournal of Scientific Computing, 2015
- Introductory Lectures on Convex Optimization2004
- Nonlinear total variation based noise removal algorithmsPhysica D: Nonlinear Phenomena, 1992
- On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operatorsMathematical Programming, 1992