Enhancing Domain Generalization in Medical Image Segmentation With Global and Local Prompts
- 1 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (11) , 19718-19732
Abstract
Enhancing domain generalization (DG) is a crucial and compelling research pursuit within the field of medical image segmentation, owing to the inherent heterogeneity observed in medical images. The recent success with large-scale pre-trained vision models (PVMs), such as Vision Transformer (ViT), inspires us to explore their application in this specific area. While a straightforward strategy involves fine-tuning the PVM using supervised signals from the source domains, this approach overlooks the domain shift issue and neglects the rich knowledge inherent in the instances themselves. To overcome these limitations, we introduce a novel framework enhanced by global and local prompts (GLPs). Specifically, to adapt PVM in the medical DG scenario, we explicitly separate domain-shared and domain-specific knowledge in the form of GLPs. Furthermore, we develop an individualized domain adapter to intricately investigate the relationship between each target domain sample and the source domains. To harness the inherent knowledge within instances, we devise two innovative regularization terms from both the consistency and anatomy perspectives, encouraging the model to preserve instance discriminability and organ position invariance. Extensive experiments and in-depth discussions in both vanilla and semi-supervised DG scenarios deriving from five diverse medical datasets consistently demonstrate the superior segmentation performance achieved by GLP. Our code and datasets are publicly available at https://github.com/xmed-lab/GLP.Keywords
Funding Information
- National Natural Science Foundation of China (62306254)
- Joint Research Scheme (JRS) through the National Natural Science Foundation of China (NSFC) and Research Grants Council (RGC) of Hong Kong (N_HKUST654/24)
This publication has 52 references indexed in Scilit:
- AViT: Adapting Vision Transformers for Small Skin Lesion Segmentation Datasets2023
- EPVT: Environment-Aware Prompt Vision Transformer for Domain Generalization in Skin Lesion Recognition2023
- Treasure in Distribution: A Domain Randomization Based Multi-source Domain Generalization for 2D Medical Image Segmentation2023
- Visual Prompt Tuning2022
- Source-Free Domain Adaptation with Contrastive Domain Alignment and Self-supervised Exploration for Face Anti-spoofing2022
- vMFNet: Compositionality Meets Domain-Generalised Segmentation2022
- MaxStyle: Adversarial Style Composition for Robust Medical Image Segmentation2022
- Semi-supervised Meta-learning with Disentanglement for Domain-Generalised Medical Image Segmentation2021
- Disentangled Representations for Domain-Generalized Cardiac Segmentation2021
- Shape-Aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains2020