Coordinate-independent sparse sufficient dimension reduction and variable selection
Authors:
- Xin ChenXCXin Chen
- Changliang ZouCZChangliang Zou
- R. Dennis CookRCR. Dennis Cook
Open Access
- 1 December 2010
- journal article
- Published by Institute of Mathematical Statistics in The Annals of Statistics
- Vol. 38 (6) , 3696-3723
Abstract
Sufficient dimension reduction (SDR) in regression, which reduces the dimension by replacing original predictors with a minimal set of their linear combinations without loss of information, is very helpful when the number of predictors is large. The standard SDR methods suffer because the estimated linear combinations usually consist of all original predictors, making it difficult to interpret. In this paper, we propose a unified method—coordinate-independent sparse estimation (CISE)—that can simultaneously achieve sparse sufficient dimension reduction and screen out irrelevant and redundant variables efficiently. CISE is subspace oriented in the sense that it incorporates a coordinate-independent penalty term with a broad series of model-based and model-free SDR approaches. This results in a Grassmann manifold optimization problem and a fast algorithm is suggested. Under mild conditions, based on manifold theories and techniques, it can be shown that CISE would perform asymptotically as well as if the true irrelevant predictors were known, which is referred to as the oracle property. Simulation studies and a real-data example demonstrate the effectiveness and efficiency of the proposed approach.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- On Consistency and Sparsity for Principal Components Analysis in High DimensionsJournal of the American Statistical Association, 2009
- Likelihood-Based Sufficient Dimension ReductionJournal of the American Statistical Association, 2009
- On General Adaptive Sparse Principal Component AnalysisJournal of Computational and Graphical Statistics, 2009
- Principal Fitted Components for Dimension Reduction in RegressionStatistical Science, 2008
- Tuning parameter selectors for the smoothly clipped absolute deviation methodBiometrika, 2007
- Fisher Lecture: Dimension Reduction in RegressionStatistical Science, 2007
- Sparse Principal Component AnalysisJournal of Computational and Graphical Statistics, 2006
- Principal Hessian Directions RevisitedJournal of the American Statistical Association, 1998
- On the Interpretation of Regression PlotsJournal of the American Statistical Association, 1994
- Sliced Inverse Regression for Dimension ReductionJournal of the American Statistical Association, 1991