Multimodal Healthcare Foundation Models: Techniques, Challenges, and Opportunities
Open Access
- 16 November 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
Abstract
With the rapid advance of multimodal foundation models, multimodal healthcare foundation models (MHFMs) are increasingly being adopted for reliable, intelligent clinical applications. They promise to advance precision medicine, streamline clinical workflows, and ultimately improve patient outcomes. Yet the healthcare domain imposes stringent requirements, including accuracy, safety, transparency, and trustworthiness, that demand techniques beyond those used for general-purpose multimodal foundation models. However, existing surveys largely take application-centric or data-oriented views and often underplay the technical underpinnings and domain-specific adaptations. To fill this gap, this survey explores and summarizes MHFMs from a technical perspective. Specifically, we provide a systematical taxonomy of MHFMs based on their methodologies, summarizing multimodal foundation model techniques that are applicable to the healthcare domain. In addition, we synthesize the key challenges and opportunities for deploying MHFMs in real-world healthcare settings, providing a roadmap for future research.This publication has 49 references indexed in Scilit:
- Knowledge-enhanced Parameter-efficient Transfer Learning with METER for medical vision-language tasksJournal of Biomedical Informatics, 2025
- A comprehensive survey of large language models and multimodal large language models in medicineInformation Fusion, 2024
- PMC-CLIP: Contrastive Language-Image Pre-training Using Biomedical Documents2023
- Masked Vision and Language Pre-training with Unimodal and Multimodal Contrastive Losses for Medical Visual Question Answering2023
- Foundation Model for Endoscopy Video Analysis via Large-Scale Self-supervised Pre-train2023
- Knowledge-enhanced visual-language pre-training on chest radiology imagesNature Communications, 2023
- SLIP: Self-supervision Meets Language-Image Pre-training2022
- Multi-modal Masked Autoencoders for Medical Vision-and-Language Pre-training2022
- Towards artificial general intelligence via a multimodal foundation modelNature Communications, 2022
- Knowledge Distillation: A SurveyInternational Journal of Computer Vision, 2021