I-MedSAM: Implicit Medical Image Segmentation with Segment Anything
Authors:
Preprint
Abstract
With the development of Deep Neural Networks (DNNs), many efforts have been made to handle medical image segmentation. Traditional methods such as nnUNet train specific segmentation models on the individual datasets. Plenty of recent methods have been proposed to adapt the foundational Segment Anything Model (SAM) to medical image segmentation. However, they still focus on discrete representations to generate pixel-wise predictions, which are spatially inflexible and scale poorly to higher resolution. In contrast, implicit methods learn continuous representations for segmentation, which is crucial for medical image segmentation. In this paper, we propose I-MedSAM, which leverages the benefits of both continuous representations and SAM, to obtain better cross-domain ability and accurate boundary delineation. Since medical image segmentation needs to predict detailed segmentation boundaries, we designed a novel adapter to enhance the SAM features with high-frequency information during Parameter Efficient Fine Tuning (PEFT). To convert the SAM features and coordinates into continuous segmentation output, we utilize Implicit Neural Representation (INR) to learn an implicit segmentation decoder. We also propose an uncertainty-guided sampling strategy for efficient learning of INR. Extensive evaluations on 2D medical image segmentation tasks have shown that our proposed method with only 1.6M trainable parameters outperforms existing methods including discrete and continuous methods. The code will be released.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Learning Implicit Feature Alignment Function for Semantic Segmentation2022
- Implicit Neural Representations for Medical Imaging Segmentation2022
- NUDF: Neural Unsigned Distance Fields for High Resolution 3D Medical Image Segmentation2022
- Introducing frequency representation into convolution neural networks for medical image segmentation via twin-Kernel Fourier convolutionComputer Methods and Programs in Biomedicine, 2021
- A Comprehensive Study on Colorectal Polyp Segmentation With ResUNet++, Conditional Random Field and Test-Time AugmentationIEEE Journal of Biomedical and Health Informatics, 2021
- nnU-Net: a self-configuring method for deep learning-based biomedical image segmentationNature Methods, 2020
- PraNet: Parallel Reverse Attention Network for Polyp Segmentation2020
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physiciansComputerized Medical Imaging and Graphics, 2015
- The neural basis of image segmentation in the primate brainNeuroscience, 2015