Teaching multimodal LLMs to comprehend 12-lead electrocardiographic images
Authors:
Top Cited Papers
Open Access
- 16 March 2026
- journal article
- research article
- Published by Springer Nature in npj Digital Medicine
- Vol. 9
Abstract
Electrocardiograms (ECGs) are essential, non-invasive diagnostic tools for assessing cardiac conditions. Existing methods often have limited generalizability, focus on narrow condition sets, and rely on raw physiological signals, which may be unavailable in resource-limited settings where only printed or digital ECG images are accessible. Recent advances in multimodal large language models (MLLMs) offer new opportunities, yet ECG image interpretation remains challenging due to the lack of instruction-tuning data and standardized benchmarks. To address these gaps, we introduce , the first large-scale ECG image instruction-tuning dataset with over one million samples, covering diverse tasks including feature recognition, rhythm analysis, morphology assessment, and clinical report generation. We develop , a fully open-source MLLM for ECG image interpretation trained on . We further curate , a human expert-developed benchmark spanning four core ECG interpretation tasks across nine datasets, incorporating both synthesized and real-world ECG images to enable clinically realistic evaluation. Our experiments demonstrate that establishes a new state of the art, outperforming general-purpose MLLMs by 21% to 33% in average accuracy. These results highlight the potential of to improve ECG image interpretation in clinical practice. All code, data and models are available at https://aimedlab.github.io/PULSE/ .Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Toward expert-level medical question answering with large language modelsNature Medicine, 2025
- Unlocking the diagnostic potential of electrocardiograms through information transfer from cardiac magnetic resonance imagingMedical Image Analysis, 2025
- How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suitesScience China Information Sciences, 2024
- Large language models encode clinical knowledgeNature, 2023
- Automated multilabel diagnosis on electrocardiographic images and signalsNature Communications, 2022
- Performance of a Convolutional Neural Network and Explainability Technique for 12-Lead Electrocardiogram InterpretationJAMA Cardiology, 2021
- Artificial intelligence-enhanced electrocardiography in cardiovascular disease managementNature Reviews Cardiology, 2021
- Data Science2020
- Tele-electrocardiography and bigdata: The CODE (Clinical Outcomes in Digital Electrocardiography) studyJournal of Electrocardiology, 2019
- Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural networkNature Medicine, 2019