Making the cut: improved ranking and selection for large-scale inference
Preprint
Abstract
Identifying leading measurement units from a large collection is a common inference task in various domains of large-scale inference. Testing approaches, which measure evidence against a null hypothesis rather than effect magnitude, tend to overpopulate lists of leading units with those associated with low measurement error. By contrast, local maximum likelihood (ML) approaches tend to favor units with high measurement error. Available Bayesian and empirical Bayesian approaches rely on specialized loss functions that result in similar deficiencies. We describe and evaluate a generic empirical Bayesian ranking procedure that populates the list of top units in a way that maximizes the expected overlap between the true and reported top lists for all list sizes. The procedure relates unit-specific posterior upper tail probabilities with their empirical distribution to yield a ranking variable. It discounts high-variance units less than popular non-ML methods and thus achieves improved operating characteristics in the models considered.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- EBSeq: an empirical Bayes hierarchical model for inference in RNA-seq experimentsBioinformatics, 2013
- Large-scale association analysis provides insights into the genetic architecture and pathophysiology of type 2 diabetesNature Genetics, 2012
- Percentile-Based Empirical Distribution Function Estimates for Performance Evaluation of Healthcare ProvidersJournal of the Royal Statistical Society Series C: Applied Statistics, 2011
- Bayesian ranking and selection methods using hierarchical mixture models in microarray studiesBiostatistics, 2009
- Confidence Intervals for Population Ranks in the Presence of Ties and Near TiesJournal of the American Statistical Association, 2009
- Testing significance relative to a fold-change threshold is a TREATBioinformatics, 2009
- A Bayesian Model for Ranking Hazardous Road SitesJournal of the Royal Statistical Society Series A: Statistics in Society, 2007
- Fundamental Differences in Cell Cycle Deregulation in Human Papillomavirus–Positive and Human Papillomavirus–Negative Head/Neck and Cervical CancersCancer Research, 2007
- On parametric empirical Bayes methods for comparing multiple groups using replicated gene expression profilesStatistics in Medicine, 2003
- Triple-goal Estimates in Two-stage Hierarchical ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 1998