Accelerated Block-coordinate Relaxation for Regularized Optimization
Author:
- Stephen J. WrightSWStephen J. Wright
- 1 January 2012
- journal article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Optimization
- Vol. 22 (1) , 159-186
Abstract
We discuss minimization of a smooth function to which is added a separable regularization function that induces structure in the solution. A block-coordinate relaxation approach with proximal linearized subproblems yields convergence to critical points, while identification of the optimal manifold (under a nondegeneracy condition) allows acceleration techniques to be applied on a reduced space. The work is motivated by experience with an algorithm for regularized logistic regression, and computational results for the algorithm on problems of this type are presented.Keywords
This publication has 20 references indexed in Scilit:
- Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm MinimizationSIAM Review, 2010
- The Group Lasso for Logistic RegressionJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- LASSO-Patternsearch algorithm with application to ophthalmology and genomic dataStatistics and Its Interface, 2008
- Fixed-Point Continuation for $\ell_1$-Minimization: Methodology and ConvergenceSIAM Journal on Optimization, 2008
- Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse ProblemsIEEE Journal of Selected Topics in Signal Processing, 2007
- A proximal method for identifying active manifoldsComputational Optimization and Applications, 2007
- Least angle regressionThe Annals of Statistics, 2004
- A simple and efficient algorithm for gene selection using sparse logistic regressionBioinformatics, 2003
- Active Sets, Nonsmoothness, and SensitivitySIAM Journal on Optimization, 2002
- Regression Shrinkage and Selection Via the LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 1996