Testing overall and moderator effects in random effects meta‐regression
- 25 January 2011
- journal article
- Published by Wiley in British Journal of Mathematical and Statistical Psychology
- Vol. 64 (1) , 1-19
Abstract
Random effects meta‐regression is a technique to synthesize results of multiple studies. It allows for a test of an overall effect, as well as for tests of effects of study characteristics, that is, (discrete or continuous) moderator effects. We describe various procedures to test moderator effects: the z , t , likelihood ratio (LR), Bartlett‐corrected LR (BcLR), and resampling tests. We compare the Type I error of these tests, and conclude that the common z test, and to a lesser extent the LR test, do not perform well since they may yield Type I error rates appreciably larger than the chosen alpha. The error rate of the resampling test is accurate, closely followed by the BcLR test. The error rate of the t test is less accurate but arguably tolerable. With respect to statistical power, the BcLR and t tests slightly outperform the resampling test. Therefore, our recommendation is to use either the resampling or the BcLR test. If these statistics are unavailable, then the t test should be used since it is certainly superior to the z test.Keywords
This publication has 41 references indexed in Scilit:
- The relationship between eating disorder not otherwise specified (EDNOS) and officially recognized eating disorders: Meta-analysis and implications for DSM.Psychological Bulletin, 2009
- Effectiveness of the Triple P Positive Parenting Program on Parenting: A Meta‐AnalysisFamily Relations, 2008
- A Re-Evaluation of Random-Effects Meta-AnalysisJournal of the Royal Statistical Society Series A: Statistics in Society, 2008
- Controlling the risk of spurious findings from meta‐regressionStatistics in Medicine, 2004
- Improved tests for a random effects meta‐regression with a single covariateStatistics in Medicine, 2003
- How should meta‐regression analyses be undertaken and interpreted?Statistics in Medicine, 2002
- Quantifying heterogeneity in a meta‐analysisStatistics in Medicine, 2002
- Advanced methods in meta‐analysis: multivariate approach and meta‐regressionStatistics in Medicine, 2002
- An empirical comparison of permutation methods for tests of partial regression coefficients in a linear modelJournal of Statistical Computation and Simulation, 1999
- Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher informationBiometrika, 1978