VSS-SAM++: Visual State Space-Aware SAM for 3-D Medical Image Segmentation

Authors:
Publisher website
Google Scholar
Abstract
The Segment Anything Model (SAM) has demonstrated groundbreaking performance in natural image segmentation, yet its direct application to medical imaging remains suboptimal due to domain shifts in data distributions and the inherent 3D nature of medical data. Although recent SAM-based methods have employed parameter-efficient transfer learning (PETL) to adapt SAM for medical tasks, they often overlook the critical 3D contextual information essential for accurate volumetric segmentation. To address this limitation, we introduce VSS-SAM++, a novel dual-branch architecture that combines SAM's foundational visual priors with Vision Mamba's capacity for modeling long-range spatial dependencies. In this framework, SAM serves as the primary encoder for high-level feature extraction, while a parallel Mamba branch captures cross-slice dependencies in 3D medical volumes. A gated hybrid attention module then dynamically fuses complementary features from both branches, adaptively weighting multi-view representations to minimize feature ambiguity and enhance segmentation precision. Extensive evaluations across nine public CT and MRI datasets demonstrate that VSS-SAM++ outperforms existing methods by 0.2-11.3% in Dice score on multi-organ and lesion segmentation tasks. The framework's robustness to domain shifts and scalability across diverse modalities highlights its potential for clinical deployment.
Funding Information
  • National Natural Science Foundation of China (62125111)
  • National Key Research and Development Program of China (2023YFB3811400)
  • International Partnership Program of Chinese Academy of Sciences (321GJHZ2023246GC)
  • High-level Talent Program in Pearl River Talent Plan of Guangdong Province (2019QN01Y986)
  • Strategic Priority Research Program of Chinese Academy of Sciences (XDB0930302)
  • Natural Science Foundation of Guangdong Province (2024A1515012138)
  • Key Laboratory for Magnetic Resonance and Multimodality Imaging of Guangdong Province (2023B1212060052)
  • Shenzhen Science and Technology Program (JCYJ20230807140514028)
  • Chongqing Special Project for Technology Innovation and Application Development (CSTB2025TIAD-KPX0008)
  • Guangdong Provincial Key Laboratory of Multimodality Non-Invasive Brain-Computer Interface (2024B1212010010)

This publication has 33 references indexed in Scilit: