Allison B. Devlin

Background: The field of clinical research informatics includes creation of clinical data repositories (CDRs) used to conduct quality improvement (QI) activities and comparative effectiveness research (CER). Ideally, CDR data are accurately and directly abstracted from disparate electronic health records (EHRs), across diverse health-systems.Objective: Investigators from Washington State’s Surgical Care Outcomes and Assessment Program (SCOAP) Comparative Effectiveness Research Translation Network (CERTAIN) are creating such a CDR. This manuscript describes the automation and validation methods used to create this digital infrastructure.Methods: SCOAP is a QI benchmarking initiative. Data are manually abstracted from EHRs and entered into a data management system. CERTAIN investigators are now deploying Caradigm’s Amalga™ tool to facilitate automated abstraction of data from multiple, disparate EHRs. Concordance is calculated to compare data automatically to manually abstracted. Performance measures are calculated between Amalga and each parent EHR. Validation takes place in repeated loops, with improvements made over time. When automated abstraction reaches the current benchmark for abstraction accuracy - 95% - itwill ‘go-live’ at each site.Progress to Date: A technical analysis was completed at 14 sites. Five sites are contributing; the remaining sites prioritized meeting Meaningful Use criteria. Participating sites are contributing 15-18 unique data feeds, totaling 13 surgical registry use cases. Common feeds are registration, laboratory, transcription/dictation, radiology, and medications. Approximately 50% of 1,320 designated data elements are being automatically abstracted – 25% from structured data; 25% from text mining.Conclusion: In semi-automating data abstraction and conducting a rigorous validation, CERTAIN investigators will semi-automate data collection to conduct QI and CER, while advancing the Learning Healthcare System.

Introduction: Delivering more appropriate, safer, and highly effective health care is the goal of a learning health care system. The Agency for Healthcare Research and Quality (AHRQ) funded enhanced registry projects: (1) to create and analyze valid data for comparative effectiveness research (CER); and (2) to enhance the ability to monitor and advance clinical quality improvement (QI). This case report describes barriers and solutions from one state-wide enhanced registry project.Methods: The Comparative Effectiveness Research and Translation Network (CERTAIN) deployed the commercially available Amalga Unified Intelligence System™ (Amalga) as a central data repository to enhance an existing QI registry (the Automation Project). An eight-step implementation process included hospital recruitment, technical electronic health record (EHR) review, hospital-specific interface planning, data ingestion, and validation. Data ownership and security protocols were established, along with formal methods to separate data management for QI purposes and research purposes. Sustainability would come from lowered chart review costs and the hospital’s desire to invest in the infrastructure after trying it.Findings: CERTAIN approached 19 hospitals in Washington State operating within 12 unaffiliated health care systems for the Automation Project. Five of the 19 completed all implementation steps. Four hospitals did not participate due to lack of perceived institutional value. Ten hospitals did not participate because their information technology (IT) departments were oversubscribed (e.g., too busy with Meaningful Use upgrades). One organization representing 22 additional hospitals expressed interest, but was unable to overcome data governance barriers in time. Questions about data use for QI versus research were resolved in a widely adopted project framework. Hospitals restricted data delivery to a subset of patients, introducing substantial technical challenges. Overcoming challenges of idiosyncratic EHR implementations required each hospital to devote more IT resources than were predicted. Cost savings did not meet projections because of the increased IT resource requirements and a different source of lowered chart review costs.Discussion: CERTAIN succeeded in recruiting unaffiliated hospitals into the Automation Project to create an enhanced registry to achieve AHRQ goals. This case report describes several distinct barriers to central data aggregation for QI and CER across unaffiliated hospitals: (1) competition for limited on-site IT expertise, (2) concerns about data use for QI versus research, (3) restrictions on data automation to a defined subset of patients, and (4) unpredictable resource needs because of idiosyncrasies among unaffiliated hospitals in how EHR data are coded, stored, and made available for transmission—even between hospitals using the same vendor’s EHR. Therefore, even a fully optimized automation infrastructure would still not achieve complete automation. The Automation Project was unable to align sufficiently with internal hospital objectives, so it could not show a compelling case for sustainability.