Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion
Top Cited Papers
Open Access
- 14 January 2025
- journal article
- research article
- Published by Springer Nature in BMC Medical Research Methodology
- Vol. 25 (1) , 1-14
Abstract
Background: The integration of real-world evidence (RWE) from real-world data (RWD) in clinical research is crucial for bridging the gap between clinical trial results and real-world outcomes. Analyzing routinely collected data to generate clinical evidence faces methodological concerns like confounding and bias, similar to prospectively documented observational studies. This study focuses on additional limitations frequently reported in the literature, providing an overview of the challenges and biases inherent to analyzing routine clinical care data, including health claims data (hereafter: routine data). Methods: We conducted a literature search on routine data studies in four high-impact journals based on the Journal Citation Reports (JCR) category “Medicine, General & Internal” as of 2022 and three oncology journals, covering articles published from January 2018 to October 2023. Articles were screened and categorized into three scenarios based on their potential to provide meaningful RWE: (1) Burden of Disease, (2) Safety and Risk Group Analysis, and (3) Treatment Comparison. Limitations of this type of data cited in the discussion sections were extracted and classified according to different bias types: main bias categories in non-randomized studies (information bias, reporting bias, selection bias, confounding) and additional routine data-specific challenges (i.e., operationalization, coding, follow-up, missing data, validation, and data quality). These classifications were then ranked by relevance in a focus group meeting of methodological experts. The search was pre-specified and registered in PROSPERO (CRD42023477616). Results: In October 2023, 227 articles were identified, 69 were assessed for eligibility, and 39 were included in the review: 11 on the burden of disease, 17 on safety and risk group analysis, and 11 on treatment comparison. Besides typical biases in observational studies, we identified additional challenges specific to RWE frequently mentioned in the discussion sections. The focus group had varied opinions on the limitations of Safety and Risk Group Analysis and Treatment Comparison but agreed on the essential limitations for the Burden of Disease category. Conclusion: This review provides a comprehensive overview of potential limitations and biases in analyzing routine data reported in recent high-impact journals. We highlighted key challenges that have high potential to impact analysis results, emphasizing the need for thorough consideration and discussion for meaningful inferences.Keywords
All Related Versions
Funding Information
- Universitätsklinikum Freiburg
This publication has 61 references indexed in Scilit:
- Association of the V122I Hereditary Transthyretin Amyloidosis Genetic Variant With Heart Failure Among Individuals of African or Hispanic/Latino AncestryJAMA, 2019
- Derivation, Validation, and Potential Treatment Implications of Novel Clinical Phenotypes for SepsisJAMA, 2019
- Cohort studies were found to be frequently biased by missing disease information due to deathJournal of Clinical Epidemiology, 2019
- On estimands and the analysis of adverse events in the presence of varying follow‐up times within the benefit assessment of therapiesPharmaceutical Statistics, 2018
- „Da kann man sich ja totklingeln, geht ja keiner ran“ – Schnittstellenprobleme zwischen stationärer, hausärztlicher und ambulant-fachspezialisierter Patientenversorgung aus Sicht Dresdner HausärzteDas Gesundheitswesen, 2018
- The importance and implications of comparator selection in pharmacoepidemiologic researchCurrent Epidemiology Reports, 2018
- When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials?Clinical Pharmacology & Therapeutics, 2017
- Real-World Evidence — What Is It and What Can It Tell Us?New England Journal of Medicine, 2016
- Electronic health records to facilitate clinical researchClinical Research in Cardiology, 2016
- Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available: Table 1.American Journal of Epidemiology, 2016