MBFTFuse: A Triple-Path Adversarial Network Based on Modality Balancing and Feature-Tracing Compensation for Infrared and Visible Image Fusion
Open Access
Abstract
Infrared and visible image fusion aims to integrate complementary information from heterogeneous images captured by different optical sensors based on distinct imaging principles; however, existing methods often exhibit modality bias, leading to weakened targets or the loss of crucial texture details. To address this, we propose MBFTFuse, an adversarial fusion network based on modality balancing and feature tracing, which consists of a triple-path generator and dual discriminators. The architecture employs a generator with a triple-path structure: a central modality-balancing path for deep feature fusion and dual edge feature-tracing paths for modality-specific enhancement. Specifically, a multi-cognitive modality-balancing module is introduced to achieve feature weight equilibrium, while a Feature-Tracing Attention Module self-enhances single-modality features to compensate for information loss in the fusion results. Furthermore, a pixel loss based on intensity histograms is designed to optimize inter-modal balance at the pixel level. Comparative experiments against nine state-of-the-art methods across three public datasets demonstrate that MBFTFuse effectively highlights infrared targets while preserving intricate visible textures. The superior performance of this method in both quantitative metrics and downstream object detection tasks contributes to extending the boundaries of sensor-driven computer vision technologies.Keywords
Funding Information
- National Natural Science Foundation of China (52201399)
- Fundamental Research Funds for the Central Universities (30924010504)
- Fundamental Research Funds for the Central Universities (30924010501)
This publication has 47 references indexed in Scilit:
- DSKFuse: Passive-active distillation learning for multi-modal image fusion via dynamic sparse kansformerExpert Systems with Applications, 2025
- LIVFusion: Luminance-optimized fusion of infrared and visible images with wavelet transformerEngineering Applications of Artificial Intelligence, 2025
- A dual-encoder network based on multi-layer feature fusion for infrared and visible image fusionInternational Journal of Machine Learning and Cybernetics, 2024
- Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion networkInformation Fusion, 2022
- RXDNFuse: A aggregated residual dense network for infrared and visible image fusionInformation Fusion, 2021
- Infrared and visible image fusion via detail preserving adversarial learningInformation Fusion, 2020
- FusionGAN: A generative adversarial network for infrared and visible image fusionInformation Fusion, 2019
- Infrared and visible image fusion methods and applications: A surveyInformation Fusion, 2019
- CBAM: Convolutional Block Attention Module2018
- Infrared and visible image fusion via gradient transfer and total variation minimizationInformation Fusion, 2016