Cross Prompting Consistency with Segment Anything Model for Semi-supervised Medical Image Segmentation
Authors:
Preprint
Abstract
Semi-supervised learning (SSL) has achieved notable progress in medical image segmentation. To achieve effective SSL, a model needs to be able to efficiently learn from limited labeled data and effectively exploiting knowledge from abundant unlabeled data. Recent developments in visual foundation models, such as the Segment Anything Model (SAM), have demonstrated remarkable adaptability with improved sample efficiency. To harness the power of foundation models for application in SSL, we propose a cross prompting consistency method with segment anything model (CPC-SAM) for semi-supervised medical image segmentation. Our method employs SAM's unique prompt design and innovates a cross-prompting strategy within a dual-branch framework to automatically generate prompts and supervisions across two decoder branches, enabling effectively learning from both scarce labeled and valuable unlabeled data. We further design a novel prompt consistency regularization, to reduce the prompt position sensitivity and to enhance the output invariance under different prompts. We validate our method on two medical image segmentation tasks. The extensive experiments with different labeled-data ratios and modalities demonstrate the superiority of our proposed method over the state-of-the-art SSL methods, with more than 9% Dice improvement on the breast cancer segmentation task.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Combining Segment Anything Model with Domain-Specific Knowledge for Semi-Supervised Learning in Medical Image Segmentation2024
- MA-SAM: Modality-agnostic SAM adaptation for 3D medical image segmentationMedical Image Analysis, 2024
- ASLSEG: Adapting Sam in the Loop for Semi-Supervised Liver Tumor Segmentation2024
- Segment Anything Model for Semi-supervised Medical Image Segmentation via Selecting Reliable Pseudo-labels2023
- Segment anything model for medical image analysis: An experimental studyMedical Image Analysis, 2023
- Mutual consistency learning for semi-supervised medical image segmentationMedical Image Analysis, 2022
- Semi-supervised medical image segmentation via uncertainty rectified pyramid consistencyMedical Image Analysis, 2022
- nnU-Net: a self-configuring method for deep learning-based biomedical image segmentationNature Methods, 2020
- Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT ImagesIEEE Transactions on Medical Imaging, 2020
- Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved?IEEE Transactions on Medical Imaging, 2018