Control of the mean number of false discoveries, Bonferroni and stability of multiple testing
Authors:
- Alexander GordonAGAlexander Gordon
- Galina GlazkoGGGalina Glazko
- Xing QiuXQXing Qiu
- Andrei YakovlevAYAndrei Yakovlev
Preprint
Abstract
The Bonferroni multiple testing procedure is commonly perceived as being overly conservative in large-scale simultaneous testing situations such as those that arise in microarray data analysis. The objective of the present study is to show that this popular belief is due to overly stringent requirements that are typically imposed on the procedure rather than to its conservative nature. To get over its notorious conservatism, we advocate using the Bonferroni selection rule as a procedure that controls the per family error rate (PFER). The present paper reports the first study of stability properties of the Bonferroni and Benjamini--Hochberg procedures. The Bonferroni procedure shows a superior stability in terms of the variance of both the number of true discoveries and the total number of discoveries, a property that is especially important in the presence of correlations between individual $p$-values. Its stability and the ability to provide strong control of the PFER make the Bonferroni procedure an attractive choice in microarray studies.Keywords
All Related Versions
- Version 1, 2007-09-04, ArXiv
- Published version: The Annals of Applied Statistics, 1 (1), 179.
This publication has 19 references indexed in Scilit:
- Assessing stability of gene selection in microarray data analysisBMC Bioinformatics, 2006
- A reanalysis of a published Affymetrix GeneChip control datasetGenome Biology, 2006
- Preferred analysis methods for Affymetrix GeneChips revealed by a wholly defined control datasetGenome Biology, 2005
- Controlling the number of false discoveries: application to high-dimensional genomic dataJournal of Statistical Planning and Inference, 2004
- Strong Control, Conservative Point Estimation and Simultaneous Conservative Consistency of False Discovery Rates: A Unified ApproachJournal of the Royal Statistical Society Series B: Statistical Methodology, 2003
- Identifying differentially expressed genes using false discovery rate controlling proceduresBioinformatics, 2003
- Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profilingCancer Cell, 2002
- Resampling-based false discovery rate controlling multiple test procedures for correlated test statisticsJournal of Statistical Planning and Inference, 1999
- Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple TestingJournal of the Royal Statistical Society Series B: Statistical Methodology, 1995
- Exploratory Data Analysis and Clinical ResearchMethods of Information in Medicine, 1982