An investigation of two multivariate permutation methods for controlling the false discovery proportion
Authors:
- 14 March 2007
- journal article
- research article
- Published by Wiley in Statistics in Medicine
- Vol. 26 (24) , 4428-4440
Abstract
Identifying genes that are differentially expressed between classes of samples is an important objective of many microarray experiments. Because of the thousands of genes typically considered, there is a tension between identifying as many of the truly differentially expressed genes as possible, but not too many genes that are not really differentially expressed (false discoveries). Controlling the proportion of identified genes that are false discoveries, the false discovery proportion (FDP), is a goal of interest. In this paper, two multivariate permutation methods are investigated for controlling the FDP. One is based on a multivariate permutation testing (MPT) method that probabilistically controls the number of false discoveries, and the other is based on the Significance Analysis of Microarrays (SAM) procedure that provides an estimate of the FDP. Both methods account for the correlations among the genes. We find the ability of the methods to control the proportion of false discoveries varies substantially depending on the implementation characteristics. For example, for both methods one can proceed from the most significant gene to the least significant gene until the estimated FDP is just above the targeted level (‘top‐down’ approach), or from the least significant gene to the most significant gene until the estimated FDP is just below the targeted level (‘bottom‐up’ approach). We find that the top‐down MPT‐based method probabilistically controls the FDP, whereas our implementation of the top‐down SAM‐based method does not. Bottom‐up MPT‐based or SAM‐based methods can result in poor control of the FDP. Published in 2007 by John Wiley & Sons, Ltd.Keywords
This publication has 16 references indexed in Scilit:
- A multiple testing procedure to associate gene expression levels with survivalStatistics in Medicine, 2005
- FDR-controlling testing procedures and sample size determination for microarraysStatistics in Medicine, 2005
- Significance testing for small microarray experimentsStatistics in Medicine, 2005
- Generalized Rank Tests for Replicated Microarray DataStatistical Applications in Genetics and Molecular Biology, 2005
- On parametric empirical Bayes methods for comparing multiple groups using replicated gene expression profilesStatistics in Medicine, 2003
- A statistical perspective on gene expression data analysisStatistics in Medicine, 2003
- Significance analysis of microarrays applied to the ionizing radiation responseProceedings of the National Academy of Sciences, 2001
- Gene-Expression Profiles in Hereditary Breast CancerNew England Journal of Medicine, 2001
- On Differential Variability of Expression Ratios: Improving Statistical Inference about Gene Expression Changes from Microarray DataJournal of Computational Biology, 2001
- Analysis of Variance for Gene Expression Microarray DataJournal of Computational Biology, 2000