Greedy Projected Gradient-Newton Method for Sparse Logistic Regression
Authors:
- 11 April 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 31 (2) , 527-538
Abstract
Sparse logistic regression (SLR), which is widely used for classification and feature selection in many fields, such as neural networks, deep learning, and bioinformatics, is the classical logistic regression model with sparsity constraints. In this paper, we perform theoretical analysis on the existence and uniqueness of the solution to the SLR, and we propose a greedy projected gradient-Newton (GPGN) method for solving the SLR. The GPGN method is a combination of the projected gradient method and the Newton method. The following characteristics show that the GPGN method achieves not only elegant theoretical results but also a remarkable numerical performance in solving the SLR: 1) the full iterative sequence generated by the GPGN method converges to a global/local minimizer of the SLR under weaker conditions; 2) the GPGN method has the properties of afinite identification for an optimal support set and local quadratic convergence; and 3) the GPGN method achieves higher accuracy and higher speed compared with a number of state-of-the-art solvers according to numerical experiments.Keywords
Funding Information
- Fundamental Research Funds for the Central Universities (2018YJS177)
- National Natural Science Foundation of China (11431002, 11571033)
This publication has 50 references indexed in Scilit:
- Sparse logistic regression with a L1/2 penalty for gene selection in cancer classificationBMC Bioinformatics, 2013
- Restricted Normal Cones and Sparsity Optimization with Affine ConstraintsFoundations of Computational Mathematics, 2013
- Face recognition via Weighted Sparse RepresentationJournal of Visual Communication and Image Representation, 2013
- Discriminant sparse neighborhood preserving embedding for face recognitionPattern Recognition, 2012
- Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methodsMathematical Programming, 2011
- Nearly unbiased variable selection under minimax concave penaltyThe Annals of Statistics, 2010
- Logistic regression for disease classification using microarray data: model selection in a largepand smallncaseBioinformatics, 2007
- Regularization and Variable Selection Via the Elastic NetJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization ProblemsAnnals of Operations Research, 2005
- A QP-free constrained Newton-type method for variational inequality problemsMathematical Programming, 1999