Low-Rank Multimodal Remote Sensing Object Detection With Frequency Filtering Experts
- 21 August 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-14
Abstract
Visible-infrared object detection for remote sensing images plays an important role in the unmanned aerial vehicle (UAV) around-the-clock application. Most of the existing work focuses on designing complex network architectures to fuse complementary features, while few methods consider computational complexity and susceptibility against modality attacks, limiting the deployment of state-of-the-art frameworks. In this paper, we present a low-rank multimodal object detection approach with frequency filtering experts, called LF-MDet, which is based on the advanced DINO (detection transformer with improved denoising anchor boxes) framework. This approach achieves more accurate detection with fewer computational resources. In particular, when the specific modality is attacked or missing, our method still maintains higher robustness towards such pervasive perturbations. Specifically, we propose a low-rank enhancement technology and a dynamic illumination-aware mask module to enable a single backbone network in the form of batch formulation to unbiasedly and compatibly extract multimodal features. Furthermore, we design a lightweight frequency expert encoder from the frequency domain perspective to efficiently fuse complementary features by filtering out amplitude noise components and mixing feature tokens. Extensive experiments are conducted on the multimodal remote sensing object detection datasets, VEDAI and DroneVehicle. The results demonstrate the superiority of the proposed approach over advanced multimodal remote sensing object detectors. Compared to the baseline method, our LF-MDet effectively reduces the floating point of operations (FLOPs) by approximately 65% while improving detection accuracy. The code is available at https://github.com/cq100/LF-MDet.Keywords
Funding Information
- National Natural Science Foundation of China (61671219)
This publication has 53 references indexed in Scilit:
- Infrared differential detection and band selection for space-based aerial targets under complex backgroundsInfrared Physics & Technology, 2024
- Deep learning-based object detection in maritime unmanned aerial vehicle imagery: Review and experimental comparisonsEngineering Applications of Artificial Intelligence, 2023
- Application of raycast method for person geolocalization and distance determination using UAV images in Real-World land search and rescue scenariosExpert Systems with Applications, 2023
- YOLO-FA: Type-1 fuzzy attention based YOLO detector for vehicle detectionExpert Systems with Applications, 2023
- An interactively reinforced paradigm for joint infrared-visible image fusion and saliency object detectionInformation Fusion, 2023
- Fourier domain structural relationship analysis for unsupervised multimodal change detectionISPRS Journal of Photogrammetry and Remote Sensing, 2023
- SGFusion: A saliency guided deep-learning framework for pixel-level image fusionInformation Fusion, 2022
- Arbitrary-Oriented Object Detection with Circular Smooth Label2020
- End-to-End Object Detection with Transformers2020
- Vehicle detection in aerial imagery : A small target detection benchmarkJournal of Visual Communication and Image Representation, 2016