SAMCT: Segment Any CT Allowing Labor-Free Task-Indicator Prompts
Authors:
Preprint
Abstract
Segment anything model (SAM), a foundation model with superior versatility and generalization across diverse segmentation tasks, has attracted widespread attention in medical imaging. However, it has been proved that SAM would encounter severe performance degradation due to the lack of medical knowledge in training and local feature encoding. Though several SAM-based models have been proposed for tuning SAM in medical imaging, they still suffer from insufficient feature extraction and highly rely on high-quality prompts. In this paper, we construct a large CT dataset consisting of 1.1M CT images and 5M masks from public datasets and propose a powerful foundation model SAMCT allowing labor-free prompts. Specifically, based on SAM, SAMCT is further equipped with a U-shaped CNN image encoder, a cross-branch interaction module, and a task-indicator prompt encoder. The U-shaped CNN image encoder works in parallel with the ViT image encoder in SAM to supplement local features. Cross-branch interaction enhances the feature expression capability of the CNN image encoder and the ViT image encoder by exchanging global perception and local features from one to the other. The task-indicator prompt encoder is a plug-and-play component to effortlessly encode task-related indicators into prompt embeddings. In this way, SAMCT can work in an automatic manner in addition to the semi-automatic interactive strategy in SAM. Extensive experiments demonstrate the superiority of SAMCT against the state-of-the-art task-specific and SAM-based medical foundation models on various tasks. The code, data, and models are released at https://github.com/xianlin7/SAMCT.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Multi-site, Multi-domain Airway Tree ModelingMedical Image Analysis, 2023
- Segment anything model for medical image analysis: An experimental studyMedical Image Analysis, 2023
- Foundation models for generalist medical artificial intelligenceNature, 2023
- Stress-testing pelvic autosegmentation algorithms using anatomical edge casesPhysics and Imaging in Radiation Oncology, 2023
- HaN‐Seg: The head and neck organ‐at‐risk CT and MR segmentation datasetMedical Physics, 2023
- The Liver Tumor Segmentation Benchmark (LiTS)Medical Image Analysis, 2022
- Rapid artificial intelligence solutions in a pandemic—The COVID-19-20 Lung CT Lesion Segmentation ChallengeMedical Image Analysis, 2022
- Toward data‐efficient learning: A benchmark for COVID‐19 CT lung and infection segmentationMedical Physics, 2020
- Evaluation of algorithms for Multi-Modality Whole Heart Segmentation: An open-access grand challengeMedical Image Analysis, 2019
- Evaluation of segmentation methods on head and neck CT: Auto‐segmentation challenge 2015Medical Physics, 2017