CMTDet: Complementary Mask-Enhanced Transformer for Multimodal Remote Sensing Object Detection
Open Access
- 3 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 19 (19391404) , 9620-9635
Abstract
Object detection based on multi-modal images is an emerging method in remote sensing which has gained extensive attention in recent years, particularly for all-weather robust detection applications. However, in multi-modal remote sensing object detection, there are issues of over-reliance on a single modality and inter-modal interference. To solve these problems, this paper proposes a Complementary Mask-enhanced Transformer Detection network (CMTDet). Two key modules are designed in CMTDet: the Dual-stage Cross-modal Interaction (DCMI) module and the Common-mode Guided Weighted Fusion (CMGF) module. The Dual-stage Cross-modal Interaction (DCMI) module consists of two components: the Differential Guided Complementary Enhancement (DGCE) sub-module and the Complementary Mask Cross-Attention (CMCA) mechanism. The DGCE component captures differential information from features of different modalities, enhancing the ability to express the semantic-channel correlation and differential information between modalities. Then, the CMCA component explores the correspondence between features of different modalities through cross-attention and reduces the dependence of feature extraction on a single modality using asymmetric complementary masks. After the improved feature extraction, the CMGF module generates guidance signals based on common-mode information, explores common information between modalities and the contribution of information across different modalities, and designs adaptive weights to retain the advantageous information of each modality. Ultimately, this enhances the semantic expression ability and robustness of the fused modality. Extensive experiments across DroneVehicle, VEDAI, and OGSOD datasets show superior performance and generalization capability. Extensive experiments on the DroneVehicle, VEDAI and OGSOD datasets show that CMTDet outperforms the baseline: it achieves 1.87% and 1.73% higher mAP50 and mAP50-95 on the DroneVehicle dataset, with a 1.98% mAP boost in nighttime scenarios, verifying its superior detection performance.Keywords
Funding Information
- National Basic Research Program of China
- Provincial Natural Science Foundation (ZR2025MS1075)
- Major Scientific and Technological Innovation Project of Shandong Province of China (2022ZLGX04)
- Major Scientific and Technological Innovation Project of Shandong Province of China (2021ZLGX05)
This publication has 51 references indexed in Scilit:
- Fine-grained classification of Orah mandarin tree plots in UAV remote sensing images based on GACL-DeepLabV3+Computers and Electronics in Agriculture, 2025
- SeaMo: A season-aware multimodal foundation model for remote sensingInformation Fusion, 2025
- Alignment-assisted Frequency Fusion Network for RGB-infrared vehicle detectionNeurocomputing, 2025
- DACFusion: Dual Asymmetric Cross-Attention guided feature fusion for multispectral object detectionNeurocomputing, 2025
- Window-Based Channel Attention for Wavelet-Enhanced Learned Image Compression2024
- Dual Attention Feature Fusion for Visible-Infrared Object Detection2023
- Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection2022
- Cross-modality attentive feature fusion for object detection in multispectral remote sensing imageryPattern Recognition, 2022
- Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems2020
- Vehicle detection in aerial imagery : A small target detection benchmarkJournal of Visual Communication and Image Representation, 2016