Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images
Open Access
- 9 April 2025
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 1-13
Abstract
Segmentation in computed tomography (CT) provides detailed anatomical information, while positron emission tomography (PET) provide the metabolic activity of cancer. Existing segmentation models in CT and PET either rely on early fusion, which struggles to effectively capture independent features from each modality, or late fusion, which is computationally expensive and fails to leverage the complementary nature of the two modalities. This research addresses the gap by proposing an intermediate fusion approach that optimally balances the strengths of both modalities. Our method leverages anatomical features to guide the fusion process while preserving spatial representation quality. We achieve this through the separate encoding of anatomical and metabolic features followed by an attentive fusion decoder. Unlike traditional fixed normalization techniques, we introduce novel “zero layers” with learnable normalization. The proposed intermediate fusion reduces the number of filters, resulting in a lightweight model. Our approach demonstrates superior performance, achieving a dice score of 0.8184 and an $$\hbox {HD}^{95}$$ score of 2.31. The implications of this study include more precise tumor delineation, leading to enhanced cancer diagnosis and more effective treatment planning.
Keywords
Funding Information
- Qatar Research Development and Innovation Council (ARG01-0522-230274)
This publication has 52 references indexed in Scilit:
- Skip-SCSE Multi-scale Attention and Co-learning Method for Oropharyngeal Tumor Segmentation on Multi-modal PET-CT Images2022
- 3D U-Net Applied to Simple Attention Module for Head and Neck Tumor Segmentation in PET and CT Images2022
- The Head and Neck Tumor Segmentation Based on 3D U-Net2022
- CCUT-Net: Pixel-Wise Global Context Channel Attention UT-Net for Head and Neck Tumor Segmentation2022
- Automatic Segmentation of Head and Neck (H&N) Primary Tumors in PET and CT Images Using 3D-Inception-ResNet Model2022
- Head and Neck Cancer Primary Tumor Auto Segmentation Using Model Ensembling of Deep Learning in PET/CT Images2022
- Automatic Head and Neck Tumor Segmentation in PET/CT with Scale Attention Network2021
- Squeeze-and-Excitation Normalization for Automated Delineation of Head and Neck Primary Tumors in Combined PET and CT Images2021
- UNet++: A Nested U-Net Architecture for Medical Image Segmentation2018
- Bridging the Gap Between 2D and 3D Organ Segmentation with Volumetric Fusion Net2018