Terminological Inconsistency and the "Fit-for-Purpose" Challenge for RWE Data Quality: Findings from a Scoping Review. (Preprint)
Authors:
- Nelio Cezar De AquinoNANelio Cezar De Aquino
- Cadiele Oliana ReichertCRCadiele Oliana Reichert
- Vinícius de Camargo CallefiVCVinícius de Camargo Callefi
- Victor Santos Duarte RamosVRVictor Santos Duarte Ramos
- Yael SznajderYSYael Sznajder
- Luis Alberto LageLLLuis Alberto Lage
- Hebert Fabricio CullerHCHebert Fabricio Culler
- Vanderson RochaVRVanderson Rocha
- Meiruze Sousa FreitasMFMeiruze Sousa Freitas
- Juliana PereiraJPJuliana Pereira
Preprint
- 15 December 2025
- preprint
- Published by JMIR Publications Inc. in JMIR Preprints
Abstract
BACKGROUND Real-World Evidence (RWE) is increasingly used for clinical and regulatory decisions. However, the quality of real-world data (RWD) (e.g., electronic health records, claims, registries) remains a challenge to credible evidence generation. OBJECTIVE The objective of this Scoping Review is to synthesize the current literature on Real-World Evidence (RWE) Data Quality (DQ), specifically focusing on identifying and mapping the scope and nature of terminological inconsistency in DQ dimensions and assessment methods, and to analyze how these challenges impact the operationalization of the 'Fit-for-Purpose' (FFP) principle in RWE data assessment. METHODS This scoping review involved a systematic search of three databases (PubMed, Web of Science, and Scopus) for publications dated between January 2013 and July 2, 2025. The search targeted documents addressing frameworks, dimensions, and assessment methods related to data quality for RWE. To ensure comprehensive coverage, the primary search was supplemented using backward citation tracking (snowballing) to identify foundational literature. Data extraction focused on quality dimensions, their context of use, and their practical application. RESULTS Ninety-one eligible documents were included, concentrated in North America and Europe. Hybrid publications (academia-industry-government collaborations) constituted the majority (54.9%). A high prevalence (43%) of reported financial conflicts of interest (COI) was identified. The most cited RWD sources were EHRs, claims, and registries. Despite lacking a standardized lexicon, Completeness was the most reported Data Quality Dimension (DQD) (78 documents), followed by Accuracy (66), Consistency (47), Compliance (43), and Reliability (42). Validity and Currentness/Timeliness (41 each), Plausibility (39), and Relevance (37) were also prominent. "Fit For Purpose" (FFP) was central (57% of documents), defined as a context-dependent judgment of data suitability for a specific secondary application. While 41% of publications cited no specific framework, the Kahn et al. harmonized framework and the U.S. FDA’s RWE Program Framework were the most prominent. CONCLUSIONS This review characterizes the RWE field as a collaborative yet fragmented ecosystem, concentrated in North America/Europe and marked by prevalent financial COIs. RWE credibility is fundamentally undermined by "terminological chaos": the lack of a standardized data quality lexicon. Key dimensions are used without consensus definitions, hindering reproducibility. This ambiguity makes the field's core FFP principle operationally unstable, as its components (e.g., reliability, relevance) also lack consensus. This subjectivity is high-risk given the prevalent COIs. A viable path forward requires moving beyond isolated frameworks toward an integrated, multi-stage solution: adopting foundational standards (e.g., ISO/IEC 25012, DAMA-DMBOK) for Upstream integrity, using Common Data Models (e.g., OMOP) for Midstream interoperability, and mandating Downstream transparency through protocol pre-registration and FAIR-aligned quality reports. CLINICALTRIAL The study protocol was registered with the Open Science Framework-OSF database under DOI 10.17605/OSF.IO/V5M86Keywords
This publication has 87 references indexed in Scilit:
- Implementing Accuracy, Completeness, and Traceability for Data ReliabilityJAMA Network Open, 2025
- Describing diversity of real world data sources in pharmacoepidemiologic studies: The DIVERSE scoping reviewPharmacoepidemiology and Drug Safety, 2024
- Regulatory and HTA Considerations for Development of Real‐World Data Derived External ControlsClinical Pharmacology & Therapeutics, 2023
- HARmonized Protocol Template to Enhance Reproducibility of hypothesis evaluating real‐world evidence studies on treatment effects: A good practices report of a joint ISPE/ISPOR task forcePharmacoepidemiology and Drug Safety, 2022
- Real‐world data: Assessing electronic health records and medical claims data to support regulatory decision‐making for drug and biological productsPharmacoepidemiology and Drug Safety, 2022
- Developing real‐world evidence from real‐world data: Transforming raw data into analytical datasetsLearning Health Systems, 2021
- STROBE Reporting Guidelines for Observational StudiesJAMA Surgery, 2021
- Conduct and reporting of citation searching in Cochrane systematic reviews: A cross‐sectional studyResearch Synthesis Methods, 2019
- Considerations in characterizing real‐world data relevance and quality for regulatory purposes: A commentaryPharmacoepidemiology and Drug Safety, 2018
- Harnessing the Power of Real‐World Evidence (RWE): A Checklist to Ensure Regulatory‐Grade Data QualityClinical Pharmacology & Therapeutics, 2017