RFTNet: Region–Attention Fusion Network Combined with Dual-Branch Vision Transformer for Multimodal Brain Tumor Image Segmentation
Open Access
- 23 December 2023
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 13 (1) , 77
Abstract
Brain tumor image segmentation plays a significant auxiliary role in clinical diagnosis. Recently, deep learning has been introduced into multimodal segmentation tasks, which construct various Convolutional Neural Network (CNN) structures to achieve excellent performance. However, most CNN-based segmentation methods have poor capability for global feature extraction. Transformer is good at modeling long-distance dependencies, but it can cause local information loss and usually has a high computational complexity. In addition, it is difficult to fully exploit the brain tumor features of different modalities. To address these issues, in this paper, we propose a region–attention fusion (RAF) network that combines a dual-branch vision Transformer (DVT), called RFTNet. In RFTNet, the DVT is exploited to capture the delicate local information and global semantics separately by two branches. Meanwhile, a novel RAF is employed to effectively fuse the images of the different modalities. Finally, we design a new hybrid loss function, called region-mixed loss function (RML) to calculate the importance of each pixel and solve the problem of class imbalance. The experiments on BrasTS2018 and BraTS2020 datasets show that our method obtains a higher segmentation accuracy than other models. Furthermore, ablation experiments prove the effectiveness of each key component in RFTNet.Keywords
Funding Information
- National Natural Science Foundation of China (62106067, 202102210170, KFJJ2021101, 2021ZKCJ14)
- key specialized research and development program of Henan Province (62106067, 202102210170, KFJJ2021101, 2021ZKCJ14)
- Open Fund Project of Key Laboratory of Grain Information Processing & Control (62106067, 202102210170, KFJJ2021101, 2021ZKCJ14)
- Innovative Funds Plan of Henan University of Technology (62106067, 202102210170, KFJJ2021101, 2021ZKCJ14)
This publication has 37 references indexed in Scilit:
- A review: Deep learning for medical image segmentation using multi-modality fusionArray, 2019
- A review on brain tumor segmentation of MRI imagesMagnetic Resonance Imaging, 2019
- HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image SegmentationIEEE Transactions on Medical Imaging, 2018
- UNet++: A Nested U-Net Architecture for Medical Image Segmentation2018
- Non-local Neural Networks2018
- Densely Connected Convolutional Networks2017
- V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation2016
- Understanding deep convolutional networksPhilosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- A survey of MRI-based medical image analysis for brain tumor studiesPhysics in Medicine & Biology, 2013