Consensus-guided latent refinement for infrared andvisible image fusion
Authors:
Preprint
- 2 September 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Infrared and visible images provide complementary cues for scene interpre-tation: infrared measurements highlight thermally salient objects, whereasvisible measurements preserve fine structures and textures. Their fusionis challenging because adaptive modality selection can amplify one sourcebefore the two modalities have formed a compatible representation. Wepropose ConRef-Fusion, a two-stage latent fusion network that first learnsa shared-decoder-compatible representation and then performs local refine-ment around this representation. In the first stage, modality-specific en-coders and a shared decoder are optimized with an equal-weight latent an-chor. In the second stage, the encoders are fixed and a lightweight consensusresidual adapter predicts a bounded correction from the latent discrepancybetween the two modalities. This design preserves a stable cross-modal ref-erence while allowing scene-dependent enhancement of thermal targets andvisible details. Experiments on the TNO, M3FD, RoadScene, and MSRSbenchmarks show that the proposed method provides competitive entropy,complementary-information, fidelity, and edge-preservation scores. Objectdetection experiments on M3FD further indicate that the fused images retaincues useful for high-level perception. Ablation and latent-decoding analyses clarify how the consensus anchor and residual refinement contribute to theobserved performanceKeywords
This publication has 20 references indexed in Scilit:
- ReFusion: Learning Image Fusion from Reconstruction with Learnable Loss Via Meta-LearningInternational Journal of Computer Vision, 2024
- LVIF-Net: Learning synchronous visible and infrared image fusion and enhancement under low-light conditionsInfrared Physics & Technology, 2024
- CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image FusionInternational Journal of Computer Vision, 2023
- Infrared and visible image fusion via mixed-frequency hierarchical guided learningInfrared Physics & Technology, 2023
- Infrared and visible image fusion with entropy-based adaptive fusion module and mask-guided convolutional neural networkInfrared Physics & Technology, 2023
- SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin TransformerIEEE/CAA Journal of Automatica Sinica, 2022
- Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion networkInformation Fusion, 2022
- FusionGAN: A generative adversarial network for infrared and visible image fusionInformation Fusion, 2019
- Infrared and visible image fusion methods and applications: A surveyInformation Fusion, 2019
- Objective image fusion performance measureElectronics Letters, 2000