Multilevel Meta-Analysis of Individual Participant Data of Single-Case Experimental Designs: One-Stage versus Two-Stage Methods
Open Access
- 30 September 2020
- journal article
- research article
- Published by Taylor & Francis in Multivariate Behavioral Research
- Vol. 57 (2-3) , 298-317
Abstract
To conduct a multilevel meta-analysis of multiple single-case experimental design (SCED) studies, the individual participant data (IPD) can be analyzed in one or two stages. In the one-stage approach, a multilevel model is estimated based on the raw data. In the two-stage approach, an effect size is calculated for each participant and these effect sizes and their sampling variances are subsequently combined to estimate a meta-analytic multilevel model. The multilevel model in the two-stage approach has fewer parameters to estimate, in exchange for the reduction of information of the raw data to effect sizes. In this paper we explore how the one-stage and two-stage IPD approaches can be applied in the context of meta-analysis of single-case designs. Both approaches are compared for several single-case designs of increasing complexity. Through a simulation study we show that the two-stage approach obtains better convergence rates for more complex models, but that model estimation does not necessarily converge at a faster speed. The point estimates of the fixed effects are unbiased for both approaches across all models, as such confirming results from methodological research on IPD meta-analysis of group-comparison designs. In light of these results, we discuss the implementation of both methods in R.Keywords
This publication has 34 references indexed in Scilit:
- Characteristics of single-case designs used to assess intervention effects in 2008Behavior Research Methods, 2011
- Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel-modeling approachesBehavior Research Methods, 2010
- Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approachesBehavior Research Methods, 2009
- A multilevel meta-analysis of single-subject experimental design studiesEvidence-Based Communication Assessment and Intervention, 2008
- Parent involvement and reading: Using curriculum‐based measurement to assess the effects of paired readingPsychology in the Schools, 2003
- Improved tests for a random effects meta‐regression with a single covariateStatistics in Medicine, 2003
- Combining single-case experimental data using hierarchical linear models.School Psychology Quarterly, 2003
- To IPD or not to IPD?Evaluation & the Health Professions, 2002
- Individual patient‐ versus group‐level data meta‐regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly headStatistics in Medicine, 2002
- On tests of the overall treatment effect in meta‐analysis with normally distributed responsesStatistics in Medicine, 2001