Variable selection with LASSO regression for complex survey data
Open Access
Abstract
Variable selection is an important step to end up with good prediction models. LASSO regression models are one of the most commonly used methods for this purpose, for which cross‐validation is the most widely applied validation technique to choose the tuning parameter . Validation techniques in a complex survey framework are closely related to “replicate weights”. However, to our knowledge, they have never been used in a LASSO regression context. Applying LASSO regression models to complex survey data could be challenging. The goal of this paper is twofold. On the one hand, we analyze the performance of replicate weights methods to select the tuning parameter for fitting LASSO regression models to complex survey data. On the other hand, we propose new replicate weights methods for the same purpose. In particular, we propose a new design‐based cross‐validation method as a combination of the traditional cross‐validation and replicate weights. The performance of all these methods has been analyzed and compared by means of an extensive simulation study to the traditional cross‐validation technique to select the tuning parameter for LASSO regression models. The results suggest a considerable improvement when the new proposal design‐based cross‐validation is used instead of the traditional cross‐validation.Keywords
Funding Information
- Agencia Estatal de Investigación (PID2020‐115882RB‐I00)
- Ministerio de Ciencia e Innovación (CEX2021‐001142‐S)
- Eusko Jaurlaritza
- Euskal Herriko Unibertsitatea
This publication has 15 references indexed in Scilit:
- Fitting Regression Models to Survey DataStatistical Science, 2017
- Model-Assisted Survey Regression Estimation with the LassoJournal of Survey Statistics and Methodology, 2017
- AIC and BIC for modeling with complex survey dataJournal of Survey Statistics and Methodology, 2015
- An Introduction to Statistical Learning2013
- Leakage in data miningACM Transactions on Knowledge Discovery From Data, 2012
- The Elements of Statistical Learning2009
- Resampling Inference with Complex Survey DataJournal of the American Statistical Association, 1988
- Fitting Logistic Models Under Case-Control or Choice Based SamplingJournal of the Royal Statistical Society Series B: Statistical Methodology, 1986
- Ignorable and informative designs in survey sampling inferenceBiometrika, 1984
- On the Variances of Asymptotically Normal Estimators from Complex SurveysInternational Statistical Review, 1983