Obtaining evidence by a single well-powered trial or several modestly powered trials
Authors:
- 14 October 2012
- journal article
- research article
- Published by SAGE Publications in Statistical Methods in Medical Research
- Vol. 25 (2) , 538-552
Abstract
There is debate whether clinical trials with suboptimal power are justified and whether results from large studies are more reliable than the (combined) results of smaller trials. We quantified the error rates for evaluations based on single conventionally powered trials (80% or 90% power) versus evaluations based on the random-effects meta-analysis of a series of smaller trials. When a treatment was assumed to have no effect but heterogeneity was present, the error rates for a single trial were increased more than 10-fold above the nominal rate, even for low heterogeneity. Conversely, for meta-analyses on a series of trials, the error rates were correct. When selective publication was present, the error rates were always increased, but they still tended to be lower for a series of trials than single trials. We conclude that evidence of efficacy based on a series of (smaller) trials, may lower the error rates compared with using a single well-powered trial. Only when both heterogeneity and selective publication can be excluded, a single trial is able to provide conclusive evidence.Keywords
This publication has 35 references indexed in Scilit:
- Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studiesBMC Medical Research Methodology, 2009
- Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional AnalysisPLOS Medicine, 2009
- Reporting Bias in Drug Trials Submitted to the Food and Drug Administration: Review of Publication and PresentationPLOS Medicine, 2008
- Publication of Clinical Trials Supporting Successful New Drug Applications: A Literature AnalysisPLOS Medicine, 2008
- Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting BiasPLOS ONE, 2008
- Reasons or excuses for avoiding meta-analysis in forest plotsBMJ, 2008
- Uncertainty in heterogeneity estimates in meta-analysesBMJ, 2007
- Contradicted and Initially Stronger Effects in Highly Cited Clinical ResearchJAMA, 2005
- Measuring inconsistency in meta-analysesBMJ, 2003
- Meta-analysis in clinical trialsControlled Clinical Trials, 1986