Efficient approximate k‐fold and leave‐one‐out cross‐validation for ridge regression
- 24 January 2013
- journal article
- research article
- Published by Wiley in Biometrical Journal
- Vol. 55 (2) , 141-155
Abstract
In model building and model evaluation, cross‐validation is a frequently used resampling method. Unfortunately, this method can be quite time consuming. In this article, we discuss an approximation method that is much faster and can be used in generalized linear models and Cox’ proportional hazards model with a ridge penalty term. Our approximation method is based on a Taylor expansion around the estimate of the full model. In this way, all cross‐validated estimates are approximated without refitting the model. The tuning parameter can now be chosen based on these approximations and can be optimized in less time. The method is most accurate when approximating leave‐one‐out cross‐validation results for large data sets which is originally the most computationally demanding situation. In order to demonstrate the method's performance, it will be applied to several microarray data sets. An R package penalized, which implements the method, is available on CRAN.Keywords
This publication has 29 references indexed in Scilit:
- A survey of cross-validation procedures for model selectionStatistics Surveys, 2010
- A comparative study of survival models for breast cancer prognostication based on microarray data: does a single gene beat them all?Bioinformatics, 2008
- Optimizing the efficiency of cross‐validation in linear discriminant analysis through selective use of the Sherman–Morrison– Woodbury inversion formulaJournal of Chemometrics, 2008
- Assessment of survival prediction models based on microarray dataBioinformatics, 2007
- Gene-expression profiles to predict distant metastasis of lymph-node-negative primary breast cancerThe Lancet, 2005
- A Gene-Expression Signature as a Predictor of Survival in Breast CancerNew England Journal of Medicine, 2002
- Generalized Cross-Validation as a Method for Choosing a Good Ridge ParameterTechnometrics, 1979
- The Relationship Between Variable Selection and Data Agumentation and a Method for PredictionTechnometrics, 1974
- An Inverse Matrix Adjustment Arising in Discriminant AnalysisThe Annals of Mathematical Statistics, 1951
- Adjustment of an Inverse Matrix Corresponding to a Change in One Element of a Given MatrixThe Annals of Mathematical Statistics, 1950