An ensemble learning method for variable selection: application to high-dimensional data and missing values
- 30 May 2022
- journal article
- research article
- Published by Taylor & Francis in Journal of Statistical Computation and Simulation
- Vol. 92 (16) , 3488-3510
Abstract
Standard approaches for variable selection in linear models are not tailored to deal properly with high-dimensional and incomplete data. Currently, methods dedicated to high-dimensional data handle missing values by ad hoc strategies, like complete case analysis or single imputation, while methods dedicated to missing values, mainly based on multiple imputation, do not discuss the imputation method to use with high-dimensional data. Consequently, both approaches appear to be limited for many modern applications. With inspiration from ensemble methods, a new variable selection method is proposed. It extends classical variable selection methods in the case of high-dimensional data with or without missing data. Theoretical properties are studied and the practical interest is demonstrated through a simulation study, as well as through an application to models specification in sequential multiple imputation. In the low-dimensional case, the procedure improves the control of the error risks, especially type I error, even without missing values for stepwise, lasso or knockoff methods. With missing values, the method performs better than reference selection methods based on multiple imputation. Similar performances are obtained in the high-dimensional case with or without missing values.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Tuning multiple imputation by predictive mean matching and local residual drawsBMC Medical Research Methodology, 2014
- Sparse least trimmed squares regression for analyzing high-dimensional large data setsThe Annals of Applied Statistics, 2013
- Missing values: sparse inverse covariance estimation and an extension to sparse regressionStatistics and Computing, 2010
- Variable selection using random forestsPattern Recognition Letters, 2010
- Stability SelectionJournal of the Royal Statistical Society Series B: Statistical Methodology, 2010
- Bootstrap model selection had similar performance for selecting authentic and noise variables compared to backward variable elimination: a simulation studyJournal of Clinical Epidemiology, 2008
- The random projection method in goodness of fit for functional dataComputational Statistics & Data Analysis, 2007
- Attribute bagging: improving accuracy of classifier ensembles by using random feature subsetsPattern Recognition, 2003
- The random subspace method for constructing decision forestsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998
- Heuristics of instability and stabilization in model selectionThe Annals of Statistics, 1996