FusionBooster: A Unified Image Fusion Boosting Paradigm
Authors:
- Chunyang ChengCCChunyang Cheng
- Tianyang XuTXTianyang Xu
- Xiao-Jun WuXWXiao-Jun Wu
- Hui LiHLHui Li
- Xi LiXLXi Li
- Josef KittlerJKJosef Kittler
Preprint
Abstract
In recent years, numerous ideas have emerged for designing a mutually reinforcing mechanism or extra stages for the image fusion task, ignoring the inevitable gaps between different vision tasks and the computational burden. We argue that there is a scope to improve the fusion performance with the help of the FusionBooster, a model specifically designed for the fusion task. In particular, our booster is based on the divide-and-conquer strategy controlled by an information probe. The booster is composed of three building blocks: the probe units, the booster layer, and the assembling module. Given the result produced by a backbone method, the probe units assess the fused image and divide the results according to their information content. This is instrumental in identifying missing information, as a step to its recovery. The recovery of the degraded components along with the fusion guidance are the role of the booster layer. Lastly, the assembling module is responsible for piecing these advanced components together to deliver the output. We use concise reconstruction loss functions in conjunction with lightweight autoencoder models to formulate the learning task, with marginal computational complexity increase. The experimental results obtained in various fusion tasks, as well as downstream detection tasks, consistently demonstrate that the proposed FusionBooster significantly improves the performance. Our code will be publicly available on the project homepage.Keywords
All Related Versions
This publication has 38 references indexed in Scilit:
- Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion networkInformation Fusion, 2022
- RFN-Nest: An end-to-end residual fusion network for infrared and visible imagesInformation Fusion, 2021
- Image fusion based on generative adversarial network consistent with perceptionInformation Fusion, 2021
- SDNet: A Versatile Squeeze-and-Decomposition Network for Real-Time Image FusionInternational Journal of Computer Vision, 2021
- RXDNFuse: A aggregated residual dense network for infrared and visible image fusionInformation Fusion, 2021
- MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusionInformation Fusion, 2021
- IFCNN: A general image fusion framework based on convolutional neural networkInformation Fusion, 2020
- FusionGAN: A generative adversarial network for infrared and visible image fusionInformation Fusion, 2019
- Infrared and visible image fusion via gradient transfer and total variation minimizationInformation Fusion, 2016
- A new image fusion performance metric based on visual information fidelityInformation Fusion, 2013