The Cluster Elastic Net for High-Dimensional Regression With Unknown Variable Grouping
Authors:
- 2 January 2014
- journal article
- research article
- Published by Taylor & Francis in Technometrics
- Vol. 56 (1) , 112-122
Abstract
In the high-dimensional regression setting, the elastic net produces a parsimonious model by shrinking all coefficients toward the origin. However, in certain settings, this behavior might not be desirable: if some features are highly correlated with each other and associated with the response, then we might wish to perform less shrinkage on the coefficients corresponding to that subset of features. We propose the cluster elastic net, which selectively shrinks the coefficients for such variables toward each other, rather than toward the origin. Instead of assuming that the clusters are known a priori, the cluster elastic net infers clusters of features from the data, on the basis of correlation among the variables as well as association with the response. These clusters are then used to more accurately perform regression. We demonstrate the theoretical advantages of our proposed approach, and explore its performance in a simulation study, and in an application to HIV drug resistance data. Supplementary materials are available online.Keywords
This publication has 25 references indexed in Scilit:
- A Sparse-Group LassoJournal of Computational and Graphical Statistics, 2013
- Variable selection and regression analysis for graph-structured covariates with an application to genomicsThe Annals of Applied Statistics, 2010
- Stability SelectionJournal of the Royal Statistical Society Series B: Statistical Methodology, 2010
- Sparse regression with exact clusteringElectronic Journal of Statistics, 2010
- Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCARBiometrics, 2008
- Genotypic predictors of human immunodeficiency virus type 1 drug resistanceProceedings of the National Academy of Sciences, 2006
- Rationale and Uses of a Public HIV Drug‐Resistance DatabaseThe Journal of Infectious Diseases, 2006
- Regularization and Variable Selection Via the Elastic NetJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Objective Criteria for the Evaluation of Clustering MethodsJournal of the American Statistical Association, 1971
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 1970