Synthesis methods used to combine observational studies and randomised trials in published meta-analyses
Authors:
Open Access
- 21 February 2024
- journal article
- research article
- Published by Springer Nature in Systematic Reviews
- Vol. 13 (1) , 1-9
Abstract
Background: This study examined the synthesis methods used in meta-analyses pooling data from observational studies (OSs) and randomised controlled trials (RCTs) from various medical disciplines. Methods: We searched Medline via PubMed to identify reports of systematic reviews of interventions, including and pooling data from RCTs and OSs published in 110 high-impact factor general and specialised journals between 2015 and 2019. Screening and data extraction were performed in duplicate. To describe the synthesis methods used in the meta-analyses, we considered the first meta-analysis presented in each article. Results: Overall, 132 reports were identified with a median number of included studies of 14 [9–26]. The median number of OSs was 6.5 [3–12] and that of RCTs was 3 [1–6]. The effect estimates recorded from OSs (i.e., adjusted or unadjusted) were not specified in 82% (n = 108) of the meta-analyses. An inverse-variance common-effect model was used in 2% (n = 3) of the meta-analyses, a random-effects model was used in 55% (n = 73), and both models were used in 40% (n = 53). A Poisson regression model was used in 1 meta-analysis, and 2 meta-analyses did not report the model they used. The mean total weight of OSs in the studied meta-analyses was 57.3% (standard deviation, ± 30.3%). Only 44 (33%) meta-analyses reported results stratified by study design. Of them, the results between OSs and RCTs had a consistent direction of effect in 70% (n = 31). Study design was explored as a potential source of heterogeneity in 79% of the meta-analyses, and confounding factors were investigated in only 10% (n = 13). Publication bias was assessed in 70% (n = 92) of the meta-analyses. Tau-square was reported in 32 meta-analyses with a median of 0.07 [0–0.30]. Conclusion: The inclusion of OSs in a meta-analysis on interventions could provide useful information. However, considerations of several methodological and conceptual aspects of OSs, that are required to avoid misleading findings, were often absent or insufficiently reported in our sample.Keywords
Funding Information
- Agence Nationale de la Recherche (ANR-22-CE36-0013-01)
This publication has 28 references indexed in Scilit:
- Emulation of Randomized Clinical Trials With Nonrandomized Database AnalysesJAMA, 2023
- Meta-analyses frequently pooled different study types together: a meta-epidemiological studyJournal of Clinical Epidemiology, 2020
- GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidenceJournal of Clinical Epidemiology, 2019
- Combining randomized and non‐randomized evidence in clinical research: a review of methods and applicationsResearch Synthesis Methods, 2014
- The Gap Between Clinical Trials and the Real WorldJAMA Internal Medicine, 2014
- Issues relating to confounding and meta‐analysis when including non‐randomized studies in systematic reviews on the effects of interventionsResearch Synthesis Methods, 2012
- Issues relating to study design and risk of bias when including non‐randomized studies in systematic reviews on the effects of interventionsResearch Synthesis Methods, 2012
- Observational studies in systemic reviews of comparative effectiveness: AHRQ and the Effective Health Care ProgramJournal of Clinical Epidemiology, 2011
- Meta-analysis of well-designed nonrandomized comparative studies of surgical procedures is as good as randomized controlled trialsJournal of Clinical Epidemiology, 2010
- Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized StudiesJAMA, 2001