Multispectral Target Detection Based on Deep Feature Fusion of Visible and Infrared Modalities
Open Access
- 23 May 2025
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 15 (11) , 5857
Abstract
Multispectral detection leverages visible and infrared imaging to improve detection performance in complex environments. However, conventional convolution-based fusion methods predominantly rely on local feature interactions, limiting their capacity to fully exploit cross-modal information and making them more susceptible to interference from complex backgrounds. To overcome these challenges, the YOLO-MEDet multispectral target detection model is proposed. Firstly, the YOLOv5 architecture is redesigned into a two-stream backbone network, incorporating a midway fusion strategy to integrate multimodal features from the C3 to C5 layers, thereby enhancing detection accuracy and robustness. Secondly, the Attention-Enhanced Feature Fusion Framework (AEFF) is introduced to optimize both cross-modal and intra-modal feature representations by employing an attention mechanism, effectively boosting model performance. Finally, the C3-PSA (C3 Pyramid Compressed Attention) module is integrated to reinforce multiscale spatial feature extraction and refine feature representation, ultimately improving detection accuracy while reducing false alarms and missed detections in complex scenarios. Extensive experiments on the FLIR, KAIST, and M3FD datasets, along with additional validation using SimuNPS simulations, confirm the superiority of YOLO-MEDet. The results indicate that the proposed model outperforms existing approaches across multiple evaluation metrics, providing an innovative solution for multispectral target detection.Keywords
Funding Information
- Liaoning Provincial Department of Science and Technology’s pilot project (2022JH24/10200029)
This publication has 52 references indexed in Scilit:
- DAMSDet: Dynamic Adaptive Multispectral Detection Transformer with Competitive Query Selection and Adaptive Feature Fusion2024
- Efficient multi-level cross-modal fusion and detection network for infrared and visible imageAlexandria Engineering Journal, 2024
- SLBAF-Net: Super-Lightweight bimodal adaptive fusion network for UAV detection in low recognition environmentMultimedia Tools and Applications, 2023
- EPSANet: An Efficient Pyramid Squeeze Attention Block on Convolutional Neural Network2023
- Multispectral satellite imagery and machine learning for the extraction of shoreline indicatorsCoastal Engineering, 2022
- RFN-Nest: An end-to-end residual fusion network for infrared and visible imagesInformation Fusion, 2021
- Adaptive spatial pixel-level feature fusion network for multispectral pedestrian detectionInfrared Physics & Technology, 2021
- Cross-modality interactive attention network for multispectral pedestrian detectionInformation Fusion, 2019
- FusionGAN: A generative adversarial network for infrared and visible image fusionInformation Fusion, 2019
- CBAM: Convolutional Block Attention Module2018