Transportability of an artificial intelligence electrocardiography model for structural heart disease: external validation and recalibration of EchoNext-Mini in MIMIC-IV

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Aims: Structural heart disease is often underdiagnosed, and echocardiography is limited by access and referral thresholds. Artificial intelligence electrocardiography (AI-ECG) may identify echocardiographically defined structural heart disease. EchoNext-Mini is a released AI-ECG model, accompanied by a benchmark dataset, that detects a composite structural heart disease outcome and component conditions from 12-lead electrocardiogram waveforms together with age, sex, and electrocardiographic measurements. We assessed its transportability and recalibration. Methods and results: We linked electrocardiograms from MIMIC-IV to subsequent transthoracic echocardiograms, including adults whose electrocardiogram preceded an eligible echocardiogram within one year, one per patient. Labels were mapped to structured echocardiographic fields using original thresholds where available, harmonized definitions otherwise. The released model was applied without retraining to this cohort and its benchmark test set, then assessed for discrimination, calibration, recalibration, subgroups, and label sensitivity. Among 45 878 patients, structural heart disease prevalence was 47.2%. Composite area under the receiver operating characteristic curve was 0.790 (95% confidence interval, 0.786 to 0.794), against 0.820 on the benchmark test set. Component values ranged from 0.679 for left ventricular wall thickness to 0.837 for right ventricular dysfunction. The composite was near-calibrated (slope, 0.93), whereas all component probabilities overestimated risk in both settings. A shift derived from the released training prevalences, requiring no local outcome data, largely corrected it. Conclusion: EchoNext-Mini transported to MIMIC-IV with composite discrimination within the published external-validation range. Component probabilities behaved as ranking scores rather than absolute risks, a property of the released model rather than the validation setting, and require adjustment before use as risk estimates.

This publication has 22 references indexed in Scilit: