Multimodal Models in Healthcare: Methods, Challenges, and Future Directions for Enhanced Clinical Decision Support
Open Access
- 10 November 2025
- journal article
- review article
- Published by MDPI AG in Information
- Vol. 16 (11) , 971
Abstract
Decision-making in modern healthcare increasingly relies on integrating a variety of data sources, including patient demographics, medical imaging, laboratory results, clinical narratives, and temporal data, all of which are difficult for traditional computational methodologies to accurately predict. This paper evaluates the latest methodologies that integrate diverse data types, including photographs, clinical notes, temporal measurements, and structured tables, through techniques such as feature amalgamation, prioritization of essential information, and utilization of graphs. We also assess pre-training, fine-tuning, and comprehensive evaluation of model generation procedures. By synthesizing findings from 50 of 91 peer-reviewed papers published between 2020 and 2024, we demonstrate that the integration of structured and unstructured data significantly improves performance in tasks like diagnosis, prognosis prediction, and personalized treatment. This review combines substantial multimodal datasets and applications across several therapeutic domains while addressing critical issues such as data heterogeneity, scalability, interpretability, and ethical considerations. This paper highlights the transformative potential of multimodal models in improving clinical decision support, providing a framework for future research to advance precision medicine and enhance healthcare outcomes.Keywords
Funding Information
- National Science Foundation (CNS-2120350, III-2311598)
This publication has 83 references indexed in Scilit:
- XRaySwinGen: Automatic medical reporting for X-ray exams with multimodal modelHeliyon, 2024
- Multimodal risk prediction with physiological signals, medical images and clinical notesHeliyon, 2024
- Development and validation of a multimodal model in predicting severe acute pancreatitis based on radiomics and deep learningInternational Journal of Medical Informatics, 2024
- Multimodal fine-tuning of clinical language models for predicting COVID-19 outcomesArtificial Intelligence in Medicine, 2023
- A survey of multimodal information fusion for smart healthcare: Mapping the journey from data to wisdomInformation Fusion, 2023
- Multimodal model with text and drug embeddings for adverse drug reaction classificationJournal of Biomedical Informatics, 2022
- An overview of deep learning methods for multimodal medical data miningExpert Systems with Applications, 2022
- Multimodal biological brain age prediction using magnetic resonance imaging and angiography with the identification of predictive regionsHuman Brain Mapping, 2022
- Knowledge representation and learning of operator clinical workflow from full-length routine fetal ultrasound scan videosMedical Image Analysis, 2021
- Assessment of a Deep Learning Model Based on Electronic Health Record Data to Forecast Clinical Outcomes in Patients With Rheumatoid ArthritisJAMA Network Open, 2019