Generalizations of the familywise error rate
Authors:
Open Access
- 1 June 2005
- journal article
- Published by Institute of Mathematical Statistics in The Annals of Statistics
- Vol. 33 (3) , 1138-1154
Abstract
Consider the problem of simultaneously testing null hypotheses H1,…,Hs. The usual approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of even one false rejection. In many applications, particularly if s is large, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which we call the k-FWER. We derive both single-step and stepdown procedures that control the k-FWER, without making any assumptions concerning the dependence structure of the p-values of the individual tests. In particular, we derive a stepdown procedure that is quite simple to apply, and prove that it cannot be improved without violation of control of the k-FWER. We also consider the false discovery proportion (FDP) defined by the number of false rejections divided by the total number of rejections (defined to be 0 if there are no rejections). The false discovery rate proposed by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289–300] controls E(FDP). Here, we construct methods such that, for any γ and α, P{FDP>γ}≤α. Two stepdown methods are proposed. The first holds under mild conditions on the dependence structure of p-values, while the second is more conservative but holds without any dependence assumptions.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- False Discovery Control for Random FieldsJournal of the American Statistical Association, 2004
- Controlling the number of false discoveries: application to high-dimensional genomic dataJournal of Statistical Planning and Inference, 2004
- A stochastic process approach to false discovery controlThe Annals of Statistics, 2004
- Multiple Testing. Part II. Step-Down Procedures for Control of the Family-Wise Error RateStatistical Applications in Genetics and Molecular Biology, 2004
- The control of the false discovery rate in multiple testing under dependencyThe Annals of Statistics, 2001
- Some probability inequalities for ordered $\rm MTP\sb 2$ random variables: a proof of the Simes conjectureThe Annals of Statistics, 1998
- The Simes Method for Multiple Hypothesis Testing with Positively Dependent Test StatisticsJournal of the American Statistical Association, 1997
- Controlled Uncertainty1988
- Multiple Comparison Procedures1987
- An improved Bonferroni procedure for multiple tests of significanceBiometrika, 1986