Philip A. Hagedorn

BACKGROUND Clinical Decision Support (CDS) systems are crucial for translating research findings into clinical practice. However, the effectiveness of CDS varies significantly, and a comprehensive evaluation framework is necessary to predict successful implementation. This study aims to address these gaps through a collaborative effort to standardize CDS evaluation across multiple sites. OBJECTIVE The objectives of this study are to (1) develop an evaluation framework for CDS across collaboration sites and (2) determine whether a collaborative CDS analytics platform improves CDS evaluation efficiency and completeness. METHODS Leveraging the EPIS (Exploration, Preparation, Implementation, Sustainment) framework, we will establish a multi-site collaborative framework for CDS evaluation. This study will develop a common evaluation framework and an innovative CDS analytics platform. Our mixed-methods approach includes forming a governance structure and workgroups to facilitate collaboration and using a modified Delphi process to develop a consensus approach to pragmatic CDS evaluation. We will then conduct a randomized, incomplete block design study to compare the efficiency and completeness of CDS evaluations across collaborating and non-collaborating sites. RESULTS As of August, 2026, the pediatric CDS collaborative includes 20 health systems and has been formalizing governance structures and standing committees. During the UG3 phase, we anticipate completing selection of eight representative CDS use cases, developing a consensus CDS evaluation framework, and designing and usability-testing a CDS evaluation dashboard and centralized analytics platform. No outcome data are available at this stage. CONCLUSIONS By establishing a standardized evaluation framework and a centralized analytics platform, the collaborative seeks to transform CDS design into a predictive science. This initiative will provide a scalable model for CDS evaluation, facilitating the rapid translation of research evidence into clinical practice across diverse healthcare settings.

Background and Objective: Diagnostic uncertainty, when unrecognized or poorly communicated, can result in diagnostic error. However, diagnostic uncertainty is challenging to study due to a lack of validated identification methods. This study aims to identify distinct linguistic patterns associated with diagnostic uncertainty in clinical documentation.Design, Setting and Participants: This case–control study compares the clinical documentation of hospitalized children who received a novel uncertain diagnosis (UD) diagnosis label during their admission to a set of matched controls. Linguistic analyses identified potential linguistic indicators (i.e., words or phrases) of diagnostic uncertainty that were then manually reviewed by a linguist and clinical experts to identify those most relevant to diagnostic uncertainty. A natural language processing program categorized medical terminology into semantic types (i.e., sign or symptom), from which we identified a subset of these semantic types that both categorized reliably and were relevant to diagnostic uncertainty. Finally, a competitive machine learning modeling strategy utilizing the linguistic indicators and semantic types compared different predictive models for identifying diagnostic uncertainty.Results: Our cohort included 242 UD‐labeled patients and 932 matched controls with a combination of 3070 clinical notes. The best‐performing model was a random forest, utilizing a combination of linguistic indicators and semantic types, yielding a sensitivity of 89.4% and a positive predictive value of 96.7%.Conclusion: Expert labeling, natural language processing, and machine learning methods combined with human validation resulted in highly predictive models to detect diagnostic uncertainty in clinical documentation and represent a promising approach to detecting, studying, and ultimately mitigating diagnostic uncertainty in clinical practice.

Background: A quality improvement initiative at our institution resulted in a new process for prospectively identifying pediatric hospital medicine (PHM) patients with uncertain diagnoses (UD). This study describes the clinical characteristics and healthcare utilization patterns of patients with UD. Methods: This single center cross-sectional study included all PHM patients identified with UD during their admission. A structured chart review was used to abstract patient demographics, primary symptoms, discharge diagnoses, and healthcare utilization patterns, including consult service use, length of stay (LOS), escalation in care, and 30-day healthcare reutilization. Appropriate descriptive statistics were used for categorical and continuous variables. Results: This study includes 200 PHM patients identified with UD. Gastrointestinal symptoms were the primary finding in 45% of patients with UD. Consult service use was highly variable, with a range of 0–8 consult services for individual patients. The median LOS was 1.6 days and only 5% required a rapid response team evaluation. As for reutilization, 7% of patients were readmitted within 30 days. Conclusions: This descriptive study highlights the heterogeneity of patients with uncertain diagnoses. Ongoing work is needed to further understand the impact of UD and to optimize the care of these patients.

*Corresponding Author: Philip A. Hagedorn, MD, MBI; E-mail: Philip.hagedorn@cchmc.org; Telephone: 513-636-0409; Twitter: @Hagedorn_MD

Objective This study attempts to characterize the inpatient communication network within a quaternary pediatric academic medical center by applying network analysis methods to secure text-messaging data. Methods We used network graphing and statistical software to create network models of an inpatient communication system with secure text-messaging data from physicians, nurses, and other ancillary staff in an academic medical center. Descriptive statistics about the network, users within the network, and visualizations informed the team's understanding of the network and its components. Results Analysis of messages exchanged over approximately 23 days revealed a large, scale-free network with 4,442 nodes and 59,913 edges. Quantitative description of user behavior (messages sent and received) and network metrics (i.e., importance of nodes within a network) revealed several operational and clinical roles both sending and receiving > 1,000 messages over this time period. While some of these nodes represented expected “dispatcher” roles in our inpatient system, others occupied important frontline clinical roles responsible for bedside clinical care. Conclusion Quantitative and network analysis of secure text-messaging logs revealed several key operational and clinical roles at risk for alert fatigue and information overload. This analysis also revealed a communication network highly reliant on these key roles, meaning disruption to these individuals or their workflows could lead to dysfunction of the communication network. While secure text-messaging applications play increasingly important roles in facilitating inpatient communication, little is understood about the impact these systems have on health care providers. Developing methods to understand and optimize communication between inpatient providers might help operational and clinical leaders to proactively prevent poorly understood pitfalls associated with these systems and build resilient and effective communication structures.

Objective Frequently overridden alerts in the electronic health record can highlight alerts that may need revision. This method is a way of fine-tuning clinical decision support. We evaluated the feasibility of a complementary, yet different method that directly involved pediatric emergency department (PED) providers in identifying additional medication alerts that were potentially incorrect or intrusive. We then evaluated the effect subsequent resulting modifications had on alert salience. Methods We performed a prospective, interventional study over 34 months (March 6, 2014, to December 31, 2016) in the PED. We implemented a passive alert feedback mechanism by enhancing the native electronic health record functionality on alert reviews. End-users flagged potentially incorrect/bothersome alerts for review by the study's team. The alerts were updated when clinically appropriate and trends of the impact were evaluated. Results More than 200 alerts were reported from both inside and outside the PED, suggesting an intuitive approach. On average, we processed 4 reviews per week from the PED, with attending physicians as major contributors. The general trend of the impact of these changes seems favorable. Discussion The implementation of the review mechanism for user-selected alerts was intuitive and sustainable and seems to be able to detect alerts that are bothersome to the end-users. The method should be run in parallel with the traditional data-driven approach to support capturing of inaccurate alerts. Conclusions User-centered, context-specific alert feedback can be used for selecting suboptimal, interruptive medication alerts.

Background: Diagnostic reasoning is an important topic in medical education, and diagnostic errors are increasingly recognized as large contributors to patient morbidity and mortality. One way to improve learner understanding of the diagnostic process is to teach the concepts of Bayesian reasoning and to make these concepts practical for clinical use. Many clinician educators do not fully understand Bayesian concepts and they lack the tools to incorporate Bayesian reasoning into clinical practice and teaching. Methods: The authors developed an interactive workshop using visual models of probabilities and thresholds, clinical cases, and available smartphone apps to teach learners about Bayesian concepts. Results: Evaluations from 3 years of workshops at a national internal medicine chief resident conference showed high satisfaction, with narrative comments suggesting learners found the visual and smartphone tools useful for applying the concepts with future learners. Conclusions: Visual models, clinical cases, and smartphone apps were well received by chief residents as a way to learn and teach Bayesian reasoning. Further study will be needed to understand if these tools can improve diagnostic accuracy or patient outcomes.

BACKGROUND AND OBJECTIVES: Despite studies indicating a high rate of overuse, electrolyte testing remains common in pediatric inpatient care. Frequently repeated electrolyte tests often return normal results and can lead to patient harm and increased cost. We aimed to reduce electrolyte testing within a hospital medicine service by >25% within 6 months. METHODS: We conducted an improvement project in which we targeted 6 hospital medicine teams at a large academic children’s hospital system by using the Model for Improvement. Interventions included standardizing communication about the electrolyte testing plan and education about the costs and risks associated with overuse of electrolyte testing. Our primary outcome measure was the number of electrolyte tests per patient day. Secondary measures included testing charges and usage rates of specific high-charge panels. We tracked medical emergency team calls and readmission rates as balancing measures. RESULTS: The mean baseline rate of electrolyte testing was 2.0 laboratory draws per 10 patient days, and this rate decreased by 35% after 1 month of initial educational interventions to 1.3 electrolyte laboratory draws per 10 patient days. This change has been sustained for 9 months and could save an estimated $292 000 in patient-level charges over the course of a year. Use of our highest-charge electrolyte panel decreased from 67% to 22% of testing. No change in rates of medical emergency team calls or readmission were found. CONCLUSIONS: Our improvement intervention was associated with significant and rapid reduction in electrolyte testing and has not been associated with unintended adverse events.