Variable selection in regression using maximal correlation and distance correlation
- 18 March 2014
- journal article
- research article
- Published by Taylor & Francis in Journal of Statistical Computation and Simulation
- Vol. 85 (8) , 1692-1705
Abstract
In most of the regression problems the first task is to select the most influential predictors explaining the response, and removing the others from the model. These problems are usually referred to as the variable selection problems in the statistical literature. Numerous methods have been proposed in this field, most of which address linear models. In this study we propose two variable selection criteria for regression based on two powerful dependence measures, maximal correlation and distance correlation. We focus on these two measures since they fully or partially satisfy the Rényi postulates for dependence measures, and thus they are able to detect nonlinear dependence structures. Therefore, our methods are considered to be appropriate in linear as well as nonlinear regression models. Both methods are easy to implement and they perform well. We illustrate the performances of the proposed methods via simulations, and compare them with two benchmark methods, stepwise Akaike information criterion and lasso. In several cases with linear dependence all four methods turned out to be comparable. In the presence of nonlinear or uncorrelated dependencies, we observed that our proposed methods may be favourable. An application of the proposed methods to a real financial data set is also provided.Keywords
This publication has 22 references indexed in Scilit:
- Feature Screening via Distance Correlation LearningJournal of the American Statistical Association, 2012
- Sure Independence Screening for Ultrahigh Dimensional Feature SpaceJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- The Adaptive Lasso and Its Oracle PropertiesJournal of the American Statistical Association, 2006
- Regularization and Variable Selection Via the Elastic NetJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Subset Selection in Regression2002
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle PropertiesJournal of the American Statistical Association, 2001
- The asymptotic distribution of the rènyi maximal correlationCommunications in Statistics - Theory and Methods, 1990
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978
- On measures of dependenceActa Mathematica Hungarica, 1959
- Das statistische Problem der Korrelation als Variations‐ und Eigenwertproblem und sein Zusammenhang mit der AusgleichsrechnungZAMM - Journal of Applied Mathematics and Mechanics / Zeitschrift für Angewandte Mathematik und Mechanik, 1941