Automatically Identifying Relevant Variables for Linear Regression with the Lasso Method: A Methodological Primer for its Application with R and a Performance Contrast Simulation with Alternative Selection Strategies
- 28 October 2019
- journal article
- research article
- Published by Taylor & Francis in Communication Methods and Measures
- Vol. 14 (3) , 204-211
Abstract
The abundance of available digital big data has created new challenges in identifying relevant variables for regression models. One statistical problem that gained relevance in the era of big data is high-dimensional statistical inference, when the number of variables greatly exceeds the number of observations. Typically, prediction errors in linear regression skyrocket when the number of included variables gets close to the number of observations, and ordinary least squares (OLS) regression no longer works in a high-dimensional scenario. Regularized estimators as a feasible solution include the Least Absolute Shrinkage and Selection Operator (Lasso), which we introduce to communication scholars here. We will include the statistical background of this technique that combines estimation and variable selection simultaneously and helps identify relevant variables for regression models in high-dimensional scenarios. We contrast the Lasso with two alternative strategies of selecting variables for regression models, namely, a theory-based “subset selection” of variables and a nonselective “all in” strategy. The simulation shows that the Lasso produces lower and more relatively stable prediction errors than the two alternative variable selection strategies, and it is therefore recommended to use, especially in high-dimensional settings typical in times of big data analysis.Keywords
This publication has 12 references indexed in Scilit:
- Evaluating Multiple Intensively Collected Media Use Measures: Validity and Reliability of Momentary AssessmentsCommunication Methods and Measures, 2015
- Big Data, Digital Media, and Computational Social ScienceThe Annals of the American Academy of Political and Social Science, 2015
- Building Better ModelsThe Annals of the American Academy of Political and Social Science, 2015
- Front MatterJournal of Economic Perspectives, 2014
- P-curve: A key to the file-drawer.Journal of Experimental Psychology: General, 2014
- Regression Shrinkage and Selection via The Lasso: A RetrospectiveJournal of the Royal Statistical Society Series B: Statistical Methodology, 2011
- Sparse modeling of categorial explanatory variablesThe Annals of Applied Statistics, 2010
- Model selection and estimation in the Gaussian graphical modelBiometrika, 2007
- Regularization and Variable Selection Via the Elastic NetJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Least angle regressionThe Annals of Statistics, 2004