ECINFusion: A Novel Explicit Channel-Wise Interaction Network for Unified Multi-Modal Medical Image Fusion
- 12 December 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 35 (5) , 4011-4025
Abstract
Multi-modal medical image fusion enhance the representation, aggregation and comprehension of functional and structural information, improving accuracy and efficiency for subsequent analysis. However, lacking explicit cross channel modeling and interaction among modalities results in the loss of details and artifacts. To this end, we propose a novel Explicit Channel-wise Interaction Network for unified multi-modal medical image Fusion, namely ECINFusion. ECINFusion encompasses two components: multi-scale adaptive feature modeling (MAFM) and explicit channel-wise interaction mechanism (ECIM). MAFM leverages adaptive parallel convolution and transformer in multi-scale manner to achieve the global context-aware feature representation. ECIM utilizes the designed multi-head channel-attention mechanism for explicit modeling in channel dimension to accomplish the cross-modal interaction. Besides, we introduce a novel adaptive L-Norm loss, preserving fine-grained details. Experiments demonstrate ECINFusion outperforms state-of-the-art approaches in various medical fusion sub-tasks on different metrics. Furthermore, extended experiments reveal the robust generalization of the proposed in different fusion tasks. In breif, the proposed explicit channel-wise interaction mechanism provides new insight for multi-modal interaction.Keywords
Funding Information
- China Postdoctoral Science Foundation (2023M731803)
- China National Postdoctoral Program for Innovative Talents (BX20230108)
- NSFC (2303173)
- NSFC (62403256)
This publication has 48 references indexed in Scilit:
- LRFNet: A real-time medical image fusion method guided by detail informationComputers in Biology and Medicine, 2024
- ADDNS: An asymmetric dual deep network with sharing mechanism for medical image fusion of CT and MR-T2Computers in Biology and Medicine, 2023
- Combining transformers with CNN for multi-focus image fusionExpert Systems with Applications, 2023
- A Systematic Literature Review on Multimodal Medical Image FusionMultimedia Tools and Applications, 2023
- Deep learning methods for medical image fusion: A reviewComputers in Biology and Medicine, 2023
- : Multimodal medical image fusion network via multi-receptive-field and multi-scale feature integrationComputers in Biology and Medicine, 2023
- A Review of the Application of Multi-modal Deep Learning in Medicine: Bibliometrics and Future DirectionsInternational Journal of Computational Intelligence Systems, 2023
- Parameter adaptive unit-linking pulse coupled neural network based MRI–PET/SPECT image fusionBiomedical Signal Processing and Control, 2023
- SDNet: A Versatile Squeeze-and-Decomposition Network for Real-Time Image FusionInternational Journal of Computer Vision, 2021
- A multiscale residual pyramid attention network for medical image fusionBiomedical Signal Processing and Control, 2021