Optimal classifier selection and negative bias in error rate estimation: an empirical study on high-dimensional prediction
Open Access
- 21 December 2009
- journal article
- research article
- Published by Springer Nature in BMC Medical Research Methodology
- Vol. 9 (1) , 85
Abstract
In biometric practice, researchers often apply a large number of different methods in a "trial-and-error" strategy to get as much as possible out of their data and, due to publication pressure or pressure from the consulting customer, present only the most favorable results. This strategy may induce a substantial optimistic bias in prediction error estimation, which is quantitatively assessed in the present manuscript. The focus of our work is on class prediction based on high-dimensional data (e.g. microarray data), since such analyses are particularly exposed to this kind of bias.Keywords
This publication has 37 references indexed in Scilit:
- A unified approach to false discovery rate estimationBMC Bioinformatics, 2008
- Almost all articles on cancer prognostic markers report statistically significant resultsEuropean Journal of Cancer, 2007
- WilcoxCV: an R package for fast variable selection in cross-validationBioinformatics, 2007
- Classification based upon gene expression data: bias and precision of error ratesBioinformatics, 2007
- Why Most Published Research Findings Are FalsePLOS Medicine, 2005
- Microarrays and molecular research: noise discovery?The Lancet, 2005
- Predictive ability of DNA microarrays for cancer outcomes and correlates: an empirical assessmentThe Lancet, 2003
- Pitfalls in the Use of DNA Microarray Data for Diagnostic and Prognostic ClassificationJNCI: Journal of the National Cancer Institute, 2003
- Selection bias in gene extraction on the basis of microarray gene-expression dataProceedings of the National Academy of Sciences, 2002
- Regression Shrinkage and Selection Via the LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 1996