Using selection models to assess sensitivity to publication bias: A tutorial and call for more routine use
Authors:
Open Access
- 1 July 2022
- journal article
- research article
- Published by SAGE Publications in Campbell Systematic Reviews
- Vol. 18 (3) , e1256
Abstract
In meta‐analyses, it is critical to assess the extent to which publication bias might have compromised the results. Classical methods based on the funnel plot, including Egger's test and Trim‐and‐Fill, have become the de facto default methods to do so, with a large majority of recent meta‐analyses in top medical journals (85%) assessing for publication bias exclusively using these methods. However, these classical funnel plot methods have important limitations when used as the sole means of assessing publication bias: they essentially assume that the publication process favors large point estimates for small studies and does not affect the largest studies, and they can perform poorly when effects are heterogeneous. In light of these limitations, we recommend that meta‐analyses routinely apply other publication bias methods in addition to or instead of classical funnel plot methods. To this end, we describe how to use and interpret selection models. These methods make the often more realistic assumption that publication bias favors “statistically significant” results, and the methods also directly accommodate effect heterogeneity. Selection models have been established for decades in the statistics literature and are supported by user‐friendly software, yet remain rarely reported in many disciplines. We use a previously published meta‐analysis to demonstrate that selection models can yield insights that extend beyond those provided by funnel plot methods, suggesting the importance of establishing more comprehensive reporting practices for publication bias assessment.Keywords
Funding Information
- NIH (R01 LM013866)
- NIH (R01 CA222147)
This publication has 45 references indexed in Scilit:
- Meta-regression approximations to reduce publication selection biasResearch Synthesis Methods, 2013
- Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trialsBMJ, 2011
- Arcsine test for publication bias in meta‐analyses with binary outcomesStatistics in Medicine, 2007
- The case of the misleading funnel plotBMJ, 2006
- Comparison of Two Methods to Detect Publication Bias in Meta-analysisJAMA, 2006
- A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpointsStatistics in Medicine, 2005
- Adjusting for publication bias in the presence of heterogeneityStatistics in Medicine, 2003
- Interpretation of significance levels and effect sizes by psychological researchers.American Psychologist, 1986
- Meta-Analytic Procedures for Social Science Research Sage Publications: Beverly Hills, 1984, 148 pp.Educational Researcher, 1986
- Estimation of Effect Size under Nonrandom Sampling: The Effects of Censoring Studies Yielding Statistically Insignificant Mean DifferencesJournal of Educational Statistics, 1984