An Orthogonal Matching Pursuit Variable Screening Algorithm for High-Dimensional Linear Regression Models
Open Access
- 1 August 2022
- journal article
- research article
- Published by Hindawi Limited in Scientific Programming
- Vol. 2022, 1-12
Abstract
Variable selection plays an important role in data mining. It is crucial to filter useful variables and extract useful information in a high-dimensional setup when the number of predictor variables d tends to be much larger than the sample size n . Statistical inferences can be more precise after irrelevant variables are moved out by the screening method. This article proposes an orthogonal matching pursuit algorithm for variable screening under the high-dimensional setup. The proposed orthogonal matching pursuit method demonstrates good performance in variable screening. In particular, if the dimension of the true model is finite, OMP might discover all relevant predictors within a finite number of steps. Throughout theoretical analysis and simulations, it is confirmed that the orthogonal matching pursuit algorithm can identify relevant predictors to ensure screening consistency in variable selection. Given the sure screening property, the BIC criterion can be used to practically select the best candidate from the models generated by the OMP algorithm. Compared with the traditional orthogonal matching pursuit method, the resulting model can improve prediction accuracy and reduce computational cost by screening out the relevant variables.Keywords
This publication has 26 references indexed in Scilit:
- Forward Regression for Ultra-High Dimensional Variable ScreeningJournal of the American Statistical Association, 2009
- Tournament screening cum EBIC for feature selection with high-dimensional feature spacesScience China Mathematics, 2009
- Covariance regularization by thresholdingThe Annals of Statistics, 2008
- Sure Independence Screening for Ultrahigh Dimensional Feature SpaceJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- Extended Bayesian information criteria for model selection with large model spacesBiometrika, 2008
- The sparsity and bias of the Lasso selection in high-dimensional linear regressionThe Annals of Statistics, 2008
- One-step sparse estimates in nonconcave penalized likelihood modelsThe Annals of Statistics, 2008
- Approximation and learning by greedy algorithmsThe Annals of Statistics, 2008
- The Adaptive Lasso and Its Oracle PropertiesJournal of the American Statistical Association, 2006
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle PropertiesJournal of the American Statistical Association, 2001