Brian Ostasiewski
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Brian Ostasiewski has a total of 56 co-authors. Here are the top co-authors by number of shared publications:
- Aaron Bridges
Clinical & Translational Science Institute, Wake Forest University School of Medicine , Winston Salem, NC 27101
3 shared publications - Adam Moses
Wake Forest Center for Healthcare Innovation, Wake Forest University School of Medicine , Winston Salem, NC 27101
3 shared publications - Ajay Dharod
Department of Internal Medicine, Department of Implementation Science, Wake Forest University School of Medicine, Winston-Salem, NC, United States
3 shared publications - Jinying Chen
Department of Medicine/Section of Preventive Medicine and Epidemiology, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, USA
3 shared publications - Kristie Long Foley
Department of Implementation Science, School of Medicine, Wake Forest University, Winston-Salem, NC, United States
3 shared publications - Sarah L. Cutrona
Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, 55 N Lake Ave, Worcester, MA, 01655, United States, 1 (774) 455-3744
3 shared publications
Latest Publications
- Open Access
- 3 July 2025
- Published by Oxford University Press (OUP) in JAMIA Open
- Vol. 8 (4)
Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling
Objectives: To develop and test a novel machine learning approach for monitoring impact of computerized clinical decision support (CDS) tools on clinicians’ electronic health record (EHR) activities. Materials and Methods: Our CDS monitoring approach leverages topic modeling, a latent-variable statistical machine learning method, to infer health providers’ EHR activities from EHR audit logs. We applied this approach to monitor the impact of a tobacco cessation support CDS tool newly implemented in 5 cancer clinics (2018-2021). We trained the topic model on EHR audit log data from 3445 encounters (pre-CDS-implementation: 1734, post-CDS-implementation: 1711) for patients with active smoking status. The number of topics was automatically determined based on within-topic coherence and across-topic divergence, and the identified topics were assigned clinically relevant EHR activity labels by 4 domain experts. Results: The topic model identified 2 distinct activities focusing on CDS (act on CDS, bypass/postpone CDS), 2 activities related to CDS (review patient records and address alerts, use note templates and acknowledge the completion of CDS), 6 related to accessing (access patient station) and reviewing patient data (external records, synopsis data, snapshot of patient data, problem list/diagnosis/notes, treatment plan), and 4 related to modifying EHR (modify diagnosis/problem lists, document visit with record review, perform administrative activities for visit and billing, and document follow-up care plan). Comparing matched 1-hour after-check-in windows post-implementation (n = 841) versus pre-implementation (n = 841) of CDS, the mean prevalence (expressed as proportions out of 1.0) of providers’ EHR-use activity increased on CDS-focused activities (0.073, 95% CI, 0.066-0.079) and CDS-related activities (0.098, 95% CI, 0.089-0.106) and decreased on modifying EHR (−0.113, 95% CI, −0.124 to −0.102) and reviewing patient data (−0.058, 95% CI, −0.072 to −0.044). Discussion: Our topic model-based CDS monitoring approach can identify shifts in prevalence of EHR-use activities pre-implementation versus post-implementation. This approach can be applied to detect unintended changes in EHR activities on a large population scale following CDS implementation, providing valuable insights to guide focused qualitative investigations for CDS improvement or de-implementation. Conclusion: Our approach offers a scalable, data-driven framework for evaluating the real-world impact of EHR-embedded CDS tools. Built on a generic machine learning framework, this approach could be adapted to explore impact of other healthcare quality improvement strategies using EHR-integrated CDS interventions.
- 3 July 2025
- Published by Oxford University Press (OUP) in JAMIA Open
- Vol. 8 (4)
Open AccessElectronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling
Objectives: To develop and test a novel machine learning approach for monitoring impact of computerized clinical decision support (CDS) tools on clinicians’ electronic health record (EHR) activities. Materials and Methods: Our CDS monitoring approach leverages topic modeling, a latent-variable statistical machine learning method, to infer health providers’ EHR activities from EHR audit logs. We applied this approach to monitor the impact of a tobacco cessation support CDS tool newly implemented in 5 cancer clinics (2018-2021). We trained the topic model on EHR audit log data from 3445 encounters (pre-CDS-implementation: 1734, post-CDS-implementation: 1711) for patients with active smoking status. The number of topics was automatically determined based on within-topic coherence and across-topic divergence, and the identified topics were assigned clinically relevant EHR activity labels by 4 domain experts. Results: The topic model identified 2 distinct activities focusing on CDS (act on CDS, bypass/postpone CDS), 2 activities related to CDS (review patient records and address alerts, use note templates and acknowledge the completion of CDS), 6 related to accessing (access patient station) and reviewing patient data (external records, synopsis data, snapshot of patient data, problem list/diagnosis/notes, treatment plan), and 4 related to modifying EHR (modify diagnosis/problem lists, document visit with record review, perform administrative activities for visit and billing, and document follow-up care plan). Comparing matched 1-hour after-check-in windows post-implementation (n = 841) versus pre-implementation (n = 841) of CDS, the mean prevalence (expressed as proportions out of 1.0) of providers’ EHR-use activity increased on CDS-focused activities (0.073, 95% CI, 0.066-0.079) and CDS-related activities (0.098, 95% CI, 0.089-0.106) and decreased on modifying EHR (−0.113, 95% CI, −0.124 to −0.102) and reviewing patient data (−0.058, 95% CI, −0.072 to −0.044). Discussion: Our topic model-based CDS monitoring approach can identify shifts in prevalence of EHR-use activities pre-implementation versus post-implementation. This approach can be applied to detect unintended changes in EHR activities on a large population scale following CDS implementation, providing valuable insights to guide focused qualitative investigations for CDS improvement or de-implementation. Conclusion: Our approach offers a scalable, data-driven framework for evaluating the real-world impact of EHR-embedded CDS tools. Built on a generic machine learning framework, this approach could be adapted to explore impact of other healthcare quality improvement strategies using EHR-integrated CDS interventions.
- Open Access
- 28 March 2023
- Published by Wiley in Learning Health Systems
- Vol. 8 (1) , e10362
Automated generation of comparator patients in the electronic medical record
Background: Well‐designed randomized trials provide high‐quality clinical evidence but are not always feasible or ethical. In their absence, the electronic medical record (EMR) presents a platform to conduct comparative effectiveness research, central to the emerging academic learning health system (aLHS) model. A barrier to realizing this vision is the lack of a process to efficiently generate a reference comparison group for each patient.Objective: To test a multi‐step process for the selection of comparators in the EMR.Materials and Methods: We conducted a mixed‐methods study within a large aLHS in North Carolina. We (1) created a list of 35 candidate variables; (2) surveyed 270 researchers to assess the importance of candidate variables; and (3) built consensus rankings around survey‐identified variables (ie, importance scores >7) across two panels of 7–8 clinical research experts. Prioritized algorithm inputs were collected from the EMR and applied using a greedy matching technique. Feasibility was measured as the percentage of patients with 100 matched comparators and performance was measured via computational time and Euclidean distance.Results: Nine variables were selected: age, sex, race, ethnicity, body mass index, insurance status, smoking status, Charlson Comorbidity Index, and neighborhood percentage in poverty. The final process successfully generated 100 matched comparators for each of 1.8 million candidate patients, executed in less than 100 min for the majority of strata, and had average Euclidean distance 0.043.Conclusion: EMR‐derived matching is feasible to implement across a diverse patient population and can provide a reproducible, efficient source of comparator data for observational studies, with additional testing in clinical research applications needed.
- 28 March 2023
- Published by Wiley in Learning Health Systems
- Vol. 8 (1) , e10362
Open AccessAutomated generation of comparator patients in the electronic medical record
Background: Well‐designed randomized trials provide high‐quality clinical evidence but are not always feasible or ethical. In their absence, the electronic medical record (EMR) presents a platform to conduct comparative effectiveness research, central to the emerging academic learning health system (aLHS) model. A barrier to realizing this vision is the lack of a process to efficiently generate a reference comparison group for each patient.Objective: To test a multi‐step process for the selection of comparators in the EMR.Materials and Methods: We conducted a mixed‐methods study within a large aLHS in North Carolina. We (1) created a list of 35 candidate variables; (2) surveyed 270 researchers to assess the importance of candidate variables; and (3) built consensus rankings around survey‐identified variables (ie, importance scores >7) across two panels of 7–8 clinical research experts. Prioritized algorithm inputs were collected from the EMR and applied using a greedy matching technique. Feasibility was measured as the percentage of patients with 100 matched comparators and performance was measured via computational time and Euclidean distance.Results: Nine variables were selected: age, sex, race, ethnicity, body mass index, insurance status, smoking status, Charlson Comorbidity Index, and neighborhood percentage in poverty. The final process successfully generated 100 matched comparators for each of 1.8 million candidate patients, executed in less than 100 min for the majority of strata, and had average Euclidean distance 0.043.Conclusion: EMR‐derived matching is feasible to implement across a diverse patient population and can provide a reproducible, efficient source of comparator data for observational studies, with additional testing in clinical research applications needed.
- Open Access
- 2 March 2023
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 11 (1) , e43097
Monitoring the Implementation of Tobacco Cessation Support Tools: Using Novel Electronic Health Record Activity Metrics
Background: Clinical decision support (CDS) tools in electronic health records (EHRs) are often used as core strategies to support quality improvement programs in the clinical setting. Monitoring the impact (intended and unintended) of these tools is crucial for program evaluation and adaptation. Existing approaches for monitoring typically rely on health care providers’ self-reports or direct observation of clinical workflows, which require substantial data collection efforts and are prone to reporting bias. Objective: This study aims to develop a novel monitoring method leveraging EHR activity data and demonstrate its use in monitoring the CDS tools implemented by a tobacco cessation program sponsored by the National Cancer Institute’s Cancer Center Cessation Initiative (C3I). Methods: We developed EHR-based metrics to monitor the implementation of two CDS tools: (1) a screening alert reminding clinic staff to complete the smoking assessment and (2) a support alert prompting health care providers to discuss support and treatment options, including referral to a cessation clinic. Using EHR activity data, we measured the completion (encounter-level alert completion rate) and burden (the number of times an alert was fired before completion and time spent handling the alert) of the CDS tools. We report metrics tracked for 12 months post implementation, comparing 7 cancer clinics (2 clinics implemented the screening alert and 5 implemented both alerts) within a C3I center, and identify areas to improve alert design and adoption. Results: The screening alert fired in 5121 encounters during the 12 months post implementation. The encounter-level alert completion rate (clinic staff acknowledged completion of screening in EHR: 0.55; clinic staff completed EHR documentation of screening results: 0.32) remained stable over time but varied considerably across clinics. The support alert fired in 1074 encounters during the 12 months. Providers acted upon (ie, not postponed) the support alert in 87.3% (n=938) of encounters, identified a patient ready to quit in 12% (n=129) of encounters, and ordered a referral to the cessation clinic in 2% (n=22) of encounters. With respect to alert burden, on average, both alerts fired over 2 times (screening alert: 2.7; support alert: 2.1) before completion; time spent postponing the screening alert was similar to completing (52 vs 53 seconds) the alert, and time spent postponing the support alert was more than completing (67 vs 50 seconds) the alert per encounter. These findings inform four areas where the alert design and use can be improved: (1) improving alert adoption and completion through local adaptation, (2) improving support alert efficacy by additional strategies including training in provider-patient communication, (3) improving the accuracy of tracking for alert completion, and (4) balancing alert efficacy with the burden. Conclusions: EHR activity metrics were able to monitor the success and burden of tobacco cessation alerts, allowing for a more nuanced understanding of potential trade-offs associated with alert implementation. These metrics can be used to guide implementation adaptation and are scalable across diverse settings.
- 2 March 2023
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 11 (1) , e43097
Open AccessMonitoring the Implementation of Tobacco Cessation Support Tools: Using Novel Electronic Health Record Activity Metrics
Background: Clinical decision support (CDS) tools in electronic health records (EHRs) are often used as core strategies to support quality improvement programs in the clinical setting. Monitoring the impact (intended and unintended) of these tools is crucial for program evaluation and adaptation. Existing approaches for monitoring typically rely on health care providers’ self-reports or direct observation of clinical workflows, which require substantial data collection efforts and are prone to reporting bias. Objective: This study aims to develop a novel monitoring method leveraging EHR activity data and demonstrate its use in monitoring the CDS tools implemented by a tobacco cessation program sponsored by the National Cancer Institute’s Cancer Center Cessation Initiative (C3I). Methods: We developed EHR-based metrics to monitor the implementation of two CDS tools: (1) a screening alert reminding clinic staff to complete the smoking assessment and (2) a support alert prompting health care providers to discuss support and treatment options, including referral to a cessation clinic. Using EHR activity data, we measured the completion (encounter-level alert completion rate) and burden (the number of times an alert was fired before completion and time spent handling the alert) of the CDS tools. We report metrics tracked for 12 months post implementation, comparing 7 cancer clinics (2 clinics implemented the screening alert and 5 implemented both alerts) within a C3I center, and identify areas to improve alert design and adoption. Results: The screening alert fired in 5121 encounters during the 12 months post implementation. The encounter-level alert completion rate (clinic staff acknowledged completion of screening in EHR: 0.55; clinic staff completed EHR documentation of screening results: 0.32) remained stable over time but varied considerably across clinics. The support alert fired in 1074 encounters during the 12 months. Providers acted upon (ie, not postponed) the support alert in 87.3% (n=938) of encounters, identified a patient ready to quit in 12% (n=129) of encounters, and ordered a referral to the cessation clinic in 2% (n=22) of encounters. With respect to alert burden, on average, both alerts fired over 2 times (screening alert: 2.7; support alert: 2.1) before completion; time spent postponing the screening alert was similar to completing (52 vs 53 seconds) the alert, and time spent postponing the support alert was more than completing (67 vs 50 seconds) the alert per encounter. These findings inform four areas where the alert design and use can be improved: (1) improving alert adoption and completion through local adaptation, (2) improving support alert efficacy by additional strategies including training in provider-patient communication, (3) improving the accuracy of tracking for alert completion, and (4) balancing alert efficacy with the burden. Conclusions: EHR activity metrics were able to monitor the success and burden of tobacco cessation alerts, allowing for a more nuanced understanding of potential trade-offs associated with alert implementation. These metrics can be used to guide implementation adaptation and are scalable across diverse settings.
- Open Access
- 29 September 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Monitoring the Implementation of Tobacco Cessation Support Tools: Using Novel Electronic Health Record Activity Metrics (Preprint)
BACKGROUND Clinical decision support (CDS) tools in electronic health records (EHRs) are often used as core strategies to support quality improvement programs in the clinical setting. Monitoring the impact (intended and unintended) of these tools is crucial for program evaluation and adaptation. Existing approaches for monitoring typically rely on health care providers’ self-reports or direct observation of clinical workflows, which require substantial data collection efforts and are prone to reporting bias. OBJECTIVE This study aims to develop a novel monitoring method leveraging EHR activity data and demonstrate its use in monitoring the CDS tools implemented by a tobacco cessation program sponsored by the National Cancer Institute’s Cancer Center Cessation Initiative (C3I). METHODS We developed EHR-based metrics to monitor the implementation of two CDS tools: (1) a screening alert reminding clinic staff to complete the smoking assessment and (2) a support alert prompting health care providers to discuss support and treatment options, including referral to a cessation clinic. Using EHR activity data, we measured the completion (encounter-level alert completion rate) and burden (the number of times an alert was fired before completion and time spent handling the alert) of the CDS tools. We report metrics tracked for 12 months post implementation, comparing 7 cancer clinics (2 clinics implemented the screening alert and 5 implemented both alerts) within a C3I center, and identify areas to improve alert design and adoption. RESULTS The screening alert fired in 5121 encounters during the 12 months post implementation. The encounter-level alert completion rate (clinic staff acknowledged completion of screening in EHR: 0.55; clinic staff completed EHR documentation of screening results: 0.32) remained stable over time but varied considerably across clinics. The support alert fired in 1074 encounters during the 12 months. Providers acted upon (ie, not postponed) the support alert in 87.3% (n=938) of encounters, identified a patient ready to quit in 12% (n=129) of encounters, and ordered a referral to the cessation clinic in 2% (n=22) of encounters. With respect to alert burden, on average, both alerts fired over 2 times (screening alert: 2.7; support alert: 2.1) before completion; time spent postponing the screening alert was similar to completing (52 vs 53 seconds) the alert, and time spent postponing the support alert was more than completing (67 vs 50 seconds) the alert per encounter. These findings inform four areas where the alert design and use can be improved: (1) improving alert adoption and completion through local adaptation, (2) improving support alert efficacy by additional strategies including training in provider-patient communication, (3) improving the accuracy of tracking for alert completion, and (4) balancing alert efficacy with the burden. CONCLUSIONS EHR activity metrics were able to monitor the success and burden of tobacco cessation alerts, allowing for a more nuanced understanding of potential trade-offs associated with alert implementation. These metrics can be used to guide implementation adaptation and are scalable across diverse settings.
- 29 September 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessMonitoring the Implementation of Tobacco Cessation Support Tools: Using Novel Electronic Health Record Activity Metrics (Preprint)
BACKGROUND Clinical decision support (CDS) tools in electronic health records (EHRs) are often used as core strategies to support quality improvement programs in the clinical setting. Monitoring the impact (intended and unintended) of these tools is crucial for program evaluation and adaptation. Existing approaches for monitoring typically rely on health care providers’ self-reports or direct observation of clinical workflows, which require substantial data collection efforts and are prone to reporting bias. OBJECTIVE This study aims to develop a novel monitoring method leveraging EHR activity data and demonstrate its use in monitoring the CDS tools implemented by a tobacco cessation program sponsored by the National Cancer Institute’s Cancer Center Cessation Initiative (C3I). METHODS We developed EHR-based metrics to monitor the implementation of two CDS tools: (1) a screening alert reminding clinic staff to complete the smoking assessment and (2) a support alert prompting health care providers to discuss support and treatment options, including referral to a cessation clinic. Using EHR activity data, we measured the completion (encounter-level alert completion rate) and burden (the number of times an alert was fired before completion and time spent handling the alert) of the CDS tools. We report metrics tracked for 12 months post implementation, comparing 7 cancer clinics (2 clinics implemented the screening alert and 5 implemented both alerts) within a C3I center, and identify areas to improve alert design and adoption. RESULTS The screening alert fired in 5121 encounters during the 12 months post implementation. The encounter-level alert completion rate (clinic staff acknowledged completion of screening in EHR: 0.55; clinic staff completed EHR documentation of screening results: 0.32) remained stable over time but varied considerably across clinics. The support alert fired in 1074 encounters during the 12 months. Providers acted upon (ie, not postponed) the support alert in 87.3% (n=938) of encounters, identified a patient ready to quit in 12% (n=129) of encounters, and ordered a referral to the cessation clinic in 2% (n=22) of encounters. With respect to alert burden, on average, both alerts fired over 2 times (screening alert: 2.7; support alert: 2.1) before completion; time spent postponing the screening alert was similar to completing (52 vs 53 seconds) the alert, and time spent postponing the support alert was more than completing (67 vs 50 seconds) the alert per encounter. These findings inform four areas where the alert design and use can be improved: (1) improving alert adoption and completion through local adaptation, (2) improving support alert efficacy by additional strategies including training in provider-patient communication, (3) improving the accuracy of tracking for alert completion, and (4) balancing alert efficacy with the burden. CONCLUSIONS EHR activity metrics were able to monitor the success and burden of tobacco cessation alerts, allowing for a more nuanced understanding of potential trade-offs associated with alert implementation. These metrics can be used to guide implementation adaptation and are scalable across diverse settings.
- Open Access
- 23 September 2022
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 10 (9) , e39746
Using Electronic Health Records for the Learning Health System: Creation of a Diabetes Research Registry
Electronic health records (EHRs) were originally developed for clinical care and billing. As such, the data are not collected, organized, and curated in a fashion that is optimized for secondary use to support the Learning Health System. Population health registries provide tools to support quality improvement. These tools are generally integrated with the live EHR, are intended to use a minimum of computing resources, and may not be appropriate for some research projects. Researchers may require different electronic phenotypes and variable definitions from those typically used for population health, and these definitions may vary from study to study. Establishing a formal registry that is mapped to the Observation Medical Outcomes Partnership common data model provides an opportunity to add custom mappings and more easily share these with other institutions. Performing preprocessing tasks such as data cleaning, calculation of risk scores, time-to-event analysis, imputation, and transforming data into a format for statistical analyses will improve efficiency and make the data easier to use for investigators. Research registries that are maintained outside the EHR also have the luxury of using significant computational resources without jeopardizing clinical care data. This paper describes a virtual Diabetes Registry at Atrium Health Wake Forest Baptist and the plan for its continued development.
- 23 September 2022
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 10 (9) , e39746
Open AccessUsing Electronic Health Records for the Learning Health System: Creation of a Diabetes Research Registry
Electronic health records (EHRs) were originally developed for clinical care and billing. As such, the data are not collected, organized, and curated in a fashion that is optimized for secondary use to support the Learning Health System. Population health registries provide tools to support quality improvement. These tools are generally integrated with the live EHR, are intended to use a minimum of computing resources, and may not be appropriate for some research projects. Researchers may require different electronic phenotypes and variable definitions from those typically used for population health, and these definitions may vary from study to study. Establishing a formal registry that is mapped to the Observation Medical Outcomes Partnership common data model provides an opportunity to add custom mappings and more easily share these with other institutions. Performing preprocessing tasks such as data cleaning, calculation of risk scores, time-to-event analysis, imputation, and transforming data into a format for statistical analyses will improve efficiency and make the data easier to use for investigators. Research registries that are maintained outside the EHR also have the luxury of using significant computational resources without jeopardizing clinical care data. This paper describes a virtual Diabetes Registry at Atrium Health Wake Forest Baptist and the plan for its continued development.
- Open Access
- 2 March 2020
- Published by American Medical Association (AMA) in JAMA Network Open
- Vol. 3 (3) , e201262
Validation of an Electronic Health Record–Based Suicide Risk Prediction Modeling Approach Across Multiple Health Care Systems
Suicide is the tenth leading cause of death in the US1 and the second most common cause of death among young US individuals.2 This important public health threat is increasing, because rates of death by suicide increased by 30% between 2000 and 2016.3 A substantially larger number of nonfatal suicide attempts occur annually, estimated to be 1.3 million cases in 2016.4 Early and accurate identification of individuals at elevated risk for suicide attempts and death is critical for developing and implementing effective strategies to prevent suicidal behavior. Although numerous factors, such as depression, substance abuse, and access to firearms, have been associated with suicide risk, they have not proven useful for individualized risk prediction. Indeed, a comprehensive meta-analysis5 of 50 years of research on longitudinal prediction of suicidal thoughts and behaviors concluded that the predictive performance of available risk factor profiles is only slightly better than chance and has not improved over the last half century.
- 2 March 2020
- Published by American Medical Association (AMA) in JAMA Network Open
- Vol. 3 (3) , e201262
Open AccessValidation of an Electronic Health Record–Based Suicide Risk Prediction Modeling Approach Across Multiple Health Care Systems
Suicide is the tenth leading cause of death in the US1 and the second most common cause of death among young US individuals.2 This important public health threat is increasing, because rates of death by suicide increased by 30% between 2000 and 2016.3 A substantially larger number of nonfatal suicide attempts occur annually, estimated to be 1.3 million cases in 2016.4 Early and accurate identification of individuals at elevated risk for suicide attempts and death is critical for developing and implementing effective strategies to prevent suicidal behavior. Although numerous factors, such as depression, substance abuse, and access to firearms, have been associated with suicide risk, they have not proven useful for individualized risk prediction. Indeed, a comprehensive meta-analysis5 of 50 years of research on longitudinal prediction of suicidal thoughts and behaviors concluded that the predictive performance of available risk factor profiles is only slightly better than chance and has not improved over the last half century.
- Open Access
- 29 March 2019
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 26 (7) , 637-645
A federated EHR network data completeness tracking system
The study sought to design, pilot, and evaluate a federated data completeness tracking system (CTX) for assessing completeness in research data extracted from electronic health record data across the Accessible Research Commons for Health (ARCH) Clinical Data Research Network. The CTX applies a systems-based approach to design workflow and technology for assessing completeness across distributed electronic health record data repositories participating in a queryable, federated network. The CTX invokes 2 positive feedback loops that utilize open source tools (DQe-c and Vue) to integrate technology and human actors in a system geared for increasing capacity and taking action. A pilot implementation of the system involved 6 ARCH partner sites between January 2017 and May 2018. The ARCH CTX has enabled the network to monitor and, if needed, adjust its data management processes to maintain complete datasets for secondary use. The system allows the network and its partner sites to profile data completeness both at the network and partner site levels. Interactive visualizations presenting the current state of completeness in the context of the entire network as well as changes in completeness across time were valued among the CTX user base. Distributed clinical data networks are complex systems. Top-down approaches that solely rely on technology to report data completeness may be necessary but not sufficient for improving completeness (and quality) of data in large-scale clinical data networks. Improving and maintaining complete (high-quality) data in such complex environments entails sociotechnical systems that exploit technology and empower human actors to engage in the process of high-quality data curating. The CTX has increased the network’s capacity to rapidly identify data completeness issues and empowered ARCH partner sites to get involved in improving the completeness of respective data in their repositories.
- 29 March 2019
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 26 (7) , 637-645
Open AccessA federated EHR network data completeness tracking system
The study sought to design, pilot, and evaluate a federated data completeness tracking system (CTX) for assessing completeness in research data extracted from electronic health record data across the Accessible Research Commons for Health (ARCH) Clinical Data Research Network. The CTX applies a systems-based approach to design workflow and technology for assessing completeness across distributed electronic health record data repositories participating in a queryable, federated network. The CTX invokes 2 positive feedback loops that utilize open source tools (DQe-c and Vue) to integrate technology and human actors in a system geared for increasing capacity and taking action. A pilot implementation of the system involved 6 ARCH partner sites between January 2017 and May 2018. The ARCH CTX has enabled the network to monitor and, if needed, adjust its data management processes to maintain complete datasets for secondary use. The system allows the network and its partner sites to profile data completeness both at the network and partner site levels. Interactive visualizations presenting the current state of completeness in the context of the entire network as well as changes in completeness across time were valued among the CTX user base. Distributed clinical data networks are complex systems. Top-down approaches that solely rely on technology to report data completeness may be necessary but not sufficient for improving completeness (and quality) of data in large-scale clinical data networks. Improving and maintaining complete (high-quality) data in such complex environments entails sociotechnical systems that exploit technology and empower human actors to engage in the process of high-quality data curating. The CTX has increased the network’s capacity to rapidly identify data completeness issues and empowered ARCH partner sites to get involved in improving the completeness of respective data in their repositories.