A test for reporting bias in trial networks: simulation and case studies
Open Access
- 27 September 2014
- journal article
- research article
- Published by Springer Nature in BMC Medical Research Methodology
- Vol. 14 (1) , 112
Abstract
Networks of trials assessing several treatment options available for the same condition are increasingly considered. Randomized trial evidence may be missing because of reporting bias. We propose a test for reporting bias in trial networks. We test whether there is an excess of trials with statistically significant results across a network of trials. The observed number of trials with nominally statistically significant p-values across the network is compared with the expected number. The performance of the test (type I error rate and power) was assessed using simulation studies under different scenarios of selective reporting bias. Examples are provided for networks of antidepressant and antipsychotic trials, where reporting biases have been previously demonstrated by comparing published to Food and Drug Administration (FDA) data. In simulations, the test maintained the type I error rate and was moderately powerful after adjustment for type I error rate, except when the between-trial variance was substantial. In all, a positive test result increased moderately or markedly the probability of reporting bias being present, while a negative test result was not very informative. In the two examples, the test gave a signal for an excess of statistically significant results in the network of published data but not in the network of FDA data. The test could be useful to document an excess of significant findings in trial networks, providing a signal for potential publication bias or other selective analysis and outcome reporting biases.Keywords
This publication has 51 references indexed in Scilit:
- Adjustment for reporting bias in network meta-analysis of antidepressant trialsBMC Medical Research Methodology, 2012
- Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic ReviewsInternational Journal of Epidemiology, 2012
- Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysisBMC Medical Research Methodology, 2011
- Network meta-analysis-highly attractive but more methodological research is neededBMC Medicine, 2011
- Comprehensive Field Synopsis and Systematic Meta-analyses of Genetic Association Studies in Cutaneous MelanomaJNCI: Journal of the National Cancer Institute, 2011
- Detecting and adjusting for small‐study effects in meta‐analysisBiometrical Journal, 2011
- Selective Publication of Antidepressant Trials and Its Influence on Apparent EfficacyNew England Journal of Medicine, 2008
- Arcsine test for publication bias in meta‐analyses with binary outcomesStatistics in Medicine, 2007
- A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpointsStatistics in Medicine, 2005
- Inflation of type I error rate in two statistical tests for the detection of publication bias in meta‐analyses with binary outcomesStatistics in Medicine, 2002