Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach
Open Access
- 24 July 2023
- journal article
- research article
- Published by Springer Nature in Behavior Research Methods
- Vol. 56 (3) , 1994-2012
Abstract
Outcome reporting bias (ORB) refers to the biasing effect caused by researchers selectively reporting outcomes within a study based on their statistical significance. ORB leads to inflated effect size estimates in meta-analysis if only the outcome with the largest effect size is reported due to ORB. We propose a new method (CORB) to correct for ORB that includes an estimate of the variability of the outcomes’ effect size as a moderator in a meta-regression model. An estimate of the variability of the outcomes’ effect size can be computed by assuming a correlation among the outcomes. Results of a Monte-Carlo simulation study showed that the effect size in meta-analyses may be severely overestimated without correcting for ORB. Estimates of CORB are close to the true effect size when overestimation caused by ORB is the largest. Applying the method to a meta-analysis on the effect of playing violent video games on aggression showed that the effect size estimate decreased when correcting for ORB. We recommend to routinely apply methods to correct for ORB in any meta-analysis. We provide annotated R code and functions to help researchers apply the CORB method.Keywords
Funding Information
- European Research Council (726361 (IMPROVE))
This publication has 73 references indexed in Scilit:
- Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated ReviewPLOS ONE, 2013
- Measuring the Prevalence of Questionable Research Practices With Incentives for Truth TellingPsychological Science, 2012
- Violent video game effects on aggression, empathy, and prosocial behavior in Eastern and Western countries: A meta-analytic review.Psychological Bulletin, 2010
- Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting BiasPLOS ONE, 2008
- A Re-Evaluation of Random-Effects Meta-AnalysisJournal of the Royal Statistical Society Series A: Statistics in Society, 2008
- Empirical Evidence for Selective Reporting of Outcomes in Randomized TrialsJAMA, 2004
- Quantifying heterogeneity in a meta‐analysisStatistics in Medicine, 2002
- Advanced methods in meta‐analysis: multivariate approach and meta‐regressionStatistics in Medicine, 2002
- Finding the missing science: The fate of studies submitted for review by a human subjects committee.Psychological Methods, 1997
- Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice VersaThe American Statistician, 1995