Multi-step adaptive elastic-net: reducing false positives in high-dimensional variable selection
- 27 February 2015
- journal article
- research article
- Published by Taylor & Francis in Journal of Statistical Computation and Simulation
- Vol. 85 (18) , 3755-3765
Abstract
Regression and variable selection in high-dimensional settings, especially when has been a popular research topic in statistical machine learning. In recent years, many successful methods have been developed to tackle this problem. In this paper, we propose the multi-step adaptive elastic-net (MSA-Enet), a multi-step estimation algorithm built upon adaptive elastic-net regularization. The numerical study on simulation data and real-world biological data sets have shown that the MSA-Enet method tends to significantly reduce the number of false-positive variables, while still maintain the estimation accuracy. By analysing the variables eliminated in each step, more insight could be gained about the structure of the correlated variable groups. These properties are desirable in many real-world variable selection and regression problems.
Keywords
Funding Information
- National Natural Science Foundation of China (11271374)
This publication has 19 references indexed in Scilit:
- Group variable selection via SCAD-L2Statistics, 2012
- Elastic SCAD as a novel penalization method for SVM classification tasks in high-dimensional dataBMC Bioinformatics, 2011
- On the adaptive elastic-net with a diverging number of parametersThe Annals of Statistics, 2009
- One-step sparse estimates in nonconcave penalized likelihood modelsThe Annals of Statistics, 2008
- L1-Regularization Path Algorithm for Generalized Linear ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2007
- The Adaptive Lasso and Its Oracle PropertiesJournal of the American Statistical Association, 2006
- Regularization and Variable Selection Via the Elastic NetJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Least angle regressionThe Annals of Statistics, 2004
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle PropertiesJournal of the American Statistical Association, 2001
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 1970