Timothy Miller

Objective: To evaluate whether retrieval-augmented generation (RAG) can serve as an efficient alternative to long-context prompting for clinical reasoning over electronic health records (EHRs). Materials and Methods: We defined 3 EHR-based tasks that are replicable across health systems and vary in reasoning complexity: (1) extracting imaging procedures (modality, date, and anatomic site), (2) generating timelines of therapeutic antibiotic use, and (3) identifying the key diagnoses for a hospitalization. Using real inpatient clinical notes from a US academic health system, we evaluated 3 large language models (GPT-5.4-mini, Mistral Medium 3, DeepSeek V3.1) with varying amounts of provided context, comparing targeted retrieval to using the most recent clinical notes. Results: For Imaging Procedures, RAG strongly outperformed recent-note inputs and exceeded long-context performance (by 0.17-9.83 F1 across all models) using fewer than 8K tokens. Similar benefits were observed for Antibiotic Timelines, where <8K of retrieved tokens matched long-context recent-notes performance (between -3.26 and +3.24 Jaccard). Error analysis revealed that missing information in the clinical notes—often due to inter-hospital transfers—limited performance to some extent. However, performance on the Diagnosis Generation task remains largely static across methods and models. Discussion: RAG demonstrated strong token efficiency across tasks, with the clearest and most consistent gains observed for imaging extraction and antibiotic timeline reconstruction. Diagnosis generation proved the most challenging task, suggesting ceiling effects imposed by documentation variability and evaluation constraints. Conclusion: Our results suggest that RAG remains a competitive and efficient approach for clinical tasks over large amounts of EHR, even as newer models become capable of handling increasingly longer amounts of text.

Background: Artificial intelligence-enhanced electrocardiography (AI-ECG) enables scalable, low-cost cardiac dysfunction screening, but existing models are annotation-intensive and predominantly adult-derived, leaving paediatric generalizability uncertain. Paediatric cohorts exhibit highly variable cardiac morphology and function compared to adults, which may be useful for learning generalizable AI-ECG models. Methods: We pretrained ECG-Fyler on a predominantly paediatric, all-age cohort at Boston Children’s Hospital (1992–2023), annotated with a cardiologyspecific coding system (Fyler codes), and evaluated it on assessments from echocardiography (echo) and cardiac magnetic resonance (CMR) studies. We validated on an external adult cohort from Columbia University Irving Medical Center. Performance was benchmarked against several AI-ECG foundation models by AUROC across age groups, lesion types, and limited-data scenarios. Findings: The pretraining cohort comprised 782,138 ECGs from 255,271 patients (median age: 10.9 years, IQR: [2.8–16.8]). Internal evaluation included 178,495 ECG-echo pairs (median age: 10.9 [3.7–17.0]) and 8,584 ECG-CMR pairs (median age: 20.7 [15.6–29.6]). External validation included 82,543 ECG-echo pairs from adults (median age: 64.0 [52.0–74.0]). ECG-Fyler improved AUROC across biventricular dysfunction and dilation tasks, with the largest gains in low-data settings. In internal validation, ECG-Fyler detected low left ventricular ejection fraction (LVEF ≤ 40%) from only 100 fine-tuning samples (AUROC: 0.80, 95% CI: [0.78–0.80]), outperforming other models (AUROC < 0.65) and improving with additional fine-tuning (AUROC: 0.94 [0.93–0.94]). Similar improvements were observed for CMR-derived LVEF, RVEF, and ventricular dilation. In external validation on adults, ECG-Fyler exhibited an AUROC of 0.83 (CI: [0.82–0.85]) for LVEF ≤ 40%. After fine-tuning on less than 10% of external data, LVEF ≤ 45% performance (AUROC: 0.87 [0.86–0.88]) outperformed a fully trained, site-specific prior model (AUROC: 0.85 [0.84–0.87]). Interpretation: Pretraining on richly annotated, paediatric-dominant ECGs yields models that transfer efficiently across institutions and ages, supporting AI-ECG screening and triage when labels or imaging access are limited. Funding: National Institutes of Health (R01LM012973); Kostin Innovation Fund, Boston Children’s Hospital. Research in context: Evidence before this study: On April 29, 2026, we searched PubMed from database inception to April 29, 2026, without language restrictions, using the terms “electrocardiogram” AND “foundation model”. We considered original studies describing ECG foundation models, transferable representation- learning approaches, or mixed-age AI-ECG studies relevant to cardiac function prediction, and excluded non-ECG studies, non-original research, and purely task-specific models without a transferable pretraining component. The available evidence consisted mainly of heterogeneous observational development and validation studies, so we did not do a formal meta-analysis. We identified 11 records. Most published artificial intelligence-enhanced electrocardiogram (AI-ECG) foundation models were developed in adult cohorts, whereas paediatric evaluation remained limited. We did not identify a previous report describing a single ECG foundation model trained on a paediatric-dominant, all-age cohort and evaluated across paediatric and adult populations using both echocardiography- and cardiac magnetic resonance-derived measures of ventricular function. Added value of this study: We developed ECG-Fyler, a clinically grounded ECG foundation model pretrained on 782,138 ECGs from 255,271 patients in a paediatric-dominant, all-age cohort using structured Fyler code annotations. We evaluated transfer learning across echocardiography- and cardiac magnetic resonance-derived tasks, congenital heart disease lesion subgroups, low-resource fine-tuning scenarios, and external adult validation. ECG-Fyler consistently outperformed training from scratch and other ECG foundation-model baselines, with the largest gains when labelled data were scarce, and showed strong cross-age and cross-institution generalization from paediatric pretraining to adult external validation. Implications of all the available evidence: Taken together, the available evidence suggests that ECG foundation models can improve data efficiency and generalizability, but the field remains dominated by adult data and task-specific applications. Our findings extend this evidence by suggesting that clinically grounded supervised pretraining on a paediatric-dominant, lifespanspanning ECG corpus can support generalizable prediction of ventricular dysfunction and dilation across age groups and institutions. If validated prospectively, such models could support lower-cost screening, triage, and longitudinal monitoring to help prioritize downstream echocardiography or cardiac magnetic resonance imaging, particularly in congenital heart disease and other settings where labelled imaging data are limited.

BACKGROUND: Delayed or missed diagnosis of congenital heart disease (CHD) contributes to excess pediatric mortality worldwide. Echocardiography (echo) is central to diagnosing and triaging CHD, yet expert interpretation remains a scarce and maldistributed global resource. Artificial intelligence offers the potential to democratize diagnostics and to extend expert-level interpretation beyond large academic centers, but its application in CHD remains underexplored. METHODS: We developed EchoFocus-CHD, an artificial intelligence–enabled model for automated detection of 12 critical and 8 noncritical CHD lesions, individually and as composites. The composite critical CHD outcome was the primary end point. The model expands on a multitask, view-agnostic architecture (PanEcho) with a transformer encoder to improve focus on relevant echo views. The model was internally trained (80%) and tested (20%) on the first echo per patient from Boston Children’s Hospital, with further evaluation on a referral cohort of echo studies performed at external US and international centers. RESULTS: The internal and referral cohorts included 3.4 million videos from 54 727 echos (median age at echo, 7.1 years [interquartile range, 0.2–15.0 years]; 5.8% critical CHD, 23.6% noncritical CHD) and 167 484 videos from 3356 echos (median age at echo, 2.5 years [interquartile range, 0.3–9.4 years]; 29.4% critical CHD, 45.6% noncritical CHD), respectively. EchoFocus-CHD showed excellent internal ability to detect the composite critical CHD outcome (area under the receiver-operating curve [AUROC], 0.94; positive likelihood ratio, 7.50; negative likelihood ratio, 0.14) and individual critical lesions (AUROC, 0.83–1.00), as well as composite noncritical CHD (AUROC, 0.90; positive likelihood ratio, 5.00; negative likelihood ratio, 0.23) and individual noncritical lesions (AUROC, 0.70–0.96). Performance declined during evaluation on the referral cohort to detect critical CHD (AUROC, 0.77), coinciding with greater expert disagreement on referral cases (κ=0.72 versus 0.82 for internal cases). Explainability analyses demonstrated that the model prioritized the same clinically relevant views (parasternal long axis, parasternal short axis, subxiphoid long axis, apical) across internal and referral cohorts, whereas uniform manifold approximation and projection analysis revealed a domain shift between cohorts. Retraining on all available US patients attenuated domain shift effects, improving international critical CHD detection (AUROC, 0.87) and calibration. CONCLUSIONS: EchoFocus-CHD shows promise for automated CHD detection to advance equitable global cardiovascular care and highlights the need to address domain shift and to establish external validation before real-world deployment.

Background: Artificial intelligence-enhanced electrocardiography (AI-ECG) enables scalable, low-cost cardiac dysfunction screening, but existing models are annotation-intensive and predominantly adult-derived, leaving paediatric generalizability uncertain. Paediatric cohorts exhibit highly variable cardiac morphology and function compared to adults, which may be useful for learning generalizable AI-ECG models.Methods: We pretrained ECG-Fyler on a predominantly paediatric, all-age cohort at Boston Children’s Hospital (1992--2023), annotated with a cardiology-specific coding system (Fyler codes), and evaluated it on assessments from echocardiography (echo) and cardiac magnetic resonance (CMR) studies. We validated on an external adult cohort from Columbia University Irving Medical Center. Performance was benchmarked against several AI-ECG foundation models by AUROC across age groups, lesion types, and limited-data scenarios.Findings: The pretraining cohort comprised 782,138 ECGs from 255,271 patients (median age: 10.9 years, IQR: [2.8--16.8]). Internal evaluation included 178,495 ECG-echo pairs (median age: 10.9 [3.7--17.0]) and 8,584 ECG-CMR pairs (median age: 20.7 [15.6--29.6]). External validation included 82,543 ECG-echo pairs from adults (median age: 64.0 [52.0--74.0]). ECG-Fyler improved AUROC compared to other approaches across biventricular dysfunction and dilation tasks, with the largest gains in low-data settings. In internal validation, ECG-Fyler detected low left ventricular ejection fraction (LVEF ≤ 40%) from only 100 fine-tuning samples (AUROC: 0.80, 95% CI: [0.78--0.80]), outperforming other models (AUROC < 0.65) and improving with additional fine-tuning (AUROC: 0.94 [0.93--0.94]). Similar improvements were observed for CMR-derived LVEF, RVEF, and ventricular dilation. In external validation on adults, ECG-Fyler exhibited an AUROC of 0.83 (CI: [0.82--0.85]) for LVEF ≤ 40%. After fine-tuning on less than 10% of external data, LVEF ≤ 45% performance (AUROC: 0.87 [0.86--0.88]) outperformed a fully trained, site-specific prior model (AUROC: 0.85 [0.84--0.87]).Interpretation: Pretraining on richly annotated, paediatric-dominant ECGs yields models that transfer efficiently across institutions and ages, supporting AI-ECG screening and triage when labels or imaging access are limited.

Recent multimodal models increasingly combine heterogeneous clinical data, yet in practice raw modalities are often replaced by expert-written summaries for convenience. However, whether such representational substitution, such as replacing medical images with their expert-written reports, preserves prognostic information has not been systematically characterized. Quantifying such gaps is essential for identifying which representational modality is most informative for predictive modeling of patient outcomes. We investigated this question by comparing the predictive utility of chest radiographs (CXRs) and their paired radiology report text for 30-day post-discharge mortality prediction using visionlanguage models (VLMs). Using the discharge note summary as global clinical context at discharge, we augmented it with either the latest pre-discharge CXR or the corresponding radiology report. On a linked MIMIC-IV/MIMIC-CXR subset with paired inputs (n = 1,360), the discharge-note-plus-CXR model achieved the best performance (AUROC = 0.864), compared with the discharge-note-only model (AUROC = 0.831) and the discharge-note-plus-report model (AUROC = 0.813). To quantify the effect of modality substitution on predictive behavior, we measured Kendall’s τ -based distances between predicted risk rankings. Inter-modality distances exceeded intra-modality distances, indicating that replacing CXRs with reports changes risk prioritization rather than merely reducing overall discrimination. Post hoc review by a radiologist further suggested that reports, as clinically oriented summaries, may not exhaustively document visually available prognostic cues that could contribute to model risk stratification. Overall, our findings suggest that expert-written reports may be imperfect prognostic proxies for raw images and that representational substitution should be evaluated using both discrimination and ranking agreement.

Objective: To evaluate the efficacy of digital twins developed using a large language model (LLaMA-3), fine-tuned with Low-Rank Adapters (LoRA) on intensive care units (ICU) physician notes, and to determine whether specialty-specific training enhances treatment recommendation accuracy compared to other ICU specialties or zero-shot baselines. Materials and Methods: Digital twins were created using LLaMA-3 fine-tuned on discharge summaries from the Medical Information Mart for Intensive Care III dataset, where medications were masked to construct training and testing datasets. The medical ICU dataset (1000 notes) was used for evaluation, and performance was assessed using Bidirectional Encoder Representations from Transformers Score (BERTScore) and ROUGE-L. A zero-shot baseline model, relying solely on contextual instructions without training, was also evaluated. While our approach moves toward digital twin capabilities, it does not incorporate real-time, patient-specific electronic health records data and can be viewed as an ICU specialty-level language model adaptation. Results: Models fine-tuned on medical ICU notes achieved the highest BERTScore (0.842), outperforming models trained on other specialties or mixed datasets. Zero-shot models showed the lowest performance, highlighting the importance of training. Discussion: The findings demonstrate that specialty-specific training significantly improves treatment recommendation accuracy in digital twins compared to generalized or zero-shot approaches. Tailoring models to specific ICU domains strengthens their clinical decision-support capabilities. Conclusion: Context-specific fine-tuning of LLMs is crucial for developing effective digital twins, offering foundational insights for personalized clinical decision support.

The TRIPOD-LLM reporting guideline for studies using large language models: a Korean translation

Background: Recognizing patient symptoms is fundamental to medicine, research, and public health. However, symptoms are often underreported in coded formats even though they are routinely documented in physician notes. Large language models (LLMs), noted for their generalizability, could help bridge this gap by mimicking the role of human expert chart reviewers for symptom identification. Objective: The primary objective of this multisite study was to measure the accurate identification of infectious respiratory disease symptoms using LLMs instructed to follow chart review guidelines. The secondary objective was to evaluate LLM generalizability in multisite settings without the need for site-specific training, fine-tuning, or customization. Methods: Four LLMs were evaluated: GPT-4, GPT-3.5, Llama2 70B, and Mixtral 8×7B. LLM prompts were instructed to take on the role of chart reviewers and follow symptom annotation guidelines when assessing physician notes. Ground truth labels for each note were annotated by subject matter experts. Optimal LLM prompting strategies were selected using a development corpus of 103 notes from the emergency department at Boston Children’s Hospital. The performance of each LLM was measured using a test corpus with 202 notes from Boston Children’s Hospital. The performance of an International Classification of Diseases, Tenth Revision (ICD-10)–based method was also measured as a baseline. Generalizability of the most performant LLM was then measured in a validation corpus of 308 notes from 21 emergency departments in the Indiana Health Information Exchange. Results: Symptom identification accuracy was superior for every LLM tested for each infectious disease symptom compared to an ICD-10–based method (F₁-score=45.1%). GPT-4 was the highest scoring (F₁-score=91.4%; P<.001) and was significantly better than the ICD-10–based method, followed by GPT-3.5 (F₁-score=90.0%; P<.001), Llama2 (F₁-score=81.7%; P<.001), and Mixtral (F₁-score=83.5%; P<.001). For the validation corpus, performance of the ICD-10–based method decreased (F₁-score=26.9%), while GPT-4 increased (F₁-score=94.0%), demonstrating better generalizability using GPT-4 (P<.001). Conclusions: LLMs significantly outperformed an ICD-10–based method for respiratory symptom identification in emergency department electronic health records. GPT-4 demonstrated the highest accuracy and generalizability, suggesting that LLMs may augment or replace traditional approaches. LLMs can be instructed to mimic human chart reviewers with high accuracy. Future work should assess broader symptom types and health care settings.