Yan Mo

The collaborative classification of hyperspectral images (HSI) with a complementary modality such as light detection and ranging (LiDAR) or synthetic aperture radar (SAR), which is known as multi-source remote sensing image classification (MRSIC), can combine rich spectral information with structural cues and is helpful for the recognition of complex ground objects. However, effectively incorporating these structural cues into multi-level spectral-spatial features and modeling their spatial relationships remain challenging under limited training samples. To address these problems, a multi-level convolutional collaborative classification network based on the DefMamba fusion mechanism (DMFNet) is proposed in this paper. First, monogenic phase congruency (MPC) responses are added to the complementary modality to enhance its structural information. Then, a dual-branch feature extraction module is constructed to derive multi-level features from HSI and the enhanced complementary modality and aggregate them into a joint representation. Finally, a DefMamba fusion module applies deformable selective scanning to the aggregated features, so as to adaptively model spatial dependencies according to the fused content. Experiments on three public multimodal remote sensing datasets, covering both HSI–LiDAR and HSI–SAR settings, demonstrate that DMFNet achieves the highest mean overall accuracy among the compared methods with 10 labeled training samples per class, with the largest improvement on the HSI-SAR scene. Additional experiments on Houston 2018 further evaluate its classification performance on a large-scale scene.

In recent years, remote sensing (RS) data have become increasingly diversified due to the continuous innovation of sensors, communications, computers, and other technologies. The use of multimodal data for Earth observation missions has become a crucial research topic. Compared with single-source RS data, the fusion of multisouce RS data can obtain more comprehensive information for categorizing scenes. However, multisource RS images fusion classification usually requires complex feature extraction and fusion, building a suitable network complexity to facilitate heterogeneous information exchange and avoid substantial redundancy is a significant challenge. To overcome these limitations, we introduce a lightweight dual-branch soft-attention classification framework, which designs the multiscale feature interaction module for collaborative HSI-LiDAR classification. Compared with other cutting-edge models, the proposed framework is compact and deeply integrates multimodality heterogeneous characteristics. The multiscale feature interaction module consists of a multiscale information fusion pattern and a soft attention module, which can effectively extract hierarchical information and improve the heterogeneous feature representation. In addition, the Transformer module adopts weight-sharing to process features from different branches, effectively improve both parameter reduction and the powerful modeling capability of long-range dependencies. To validate the efficacy and advantages of our proposed framework, extensive experiments were performed across three datasets. The results indicate superior performance compared to current state-of-the-art approaches.

Crafting an edge-based real-time object detector for unmanned aerial vehicle (UAV) aerial images is challenging because of the limited computational resources and the small size of detected objects. Existing lightweight object detectors often prioritize speed over detecting extremely small targets. To better balance this trade-off, this paper proposes an efficient and low-complexity object detector for edge computing platforms deployed on UAVs, termed EUAVDet (Edge-based UAV Object Detector). Specifically, an efficient feature downsampling module and a novel multi-kernel aggregation block are first introduced into the backbone network to retain more feature details and capture richer spatial information. Subsequently, an improved feature pyramid network with a faster ghost module is incorporated into the neck network to fuse multi-scale features with fewer parameters. Experimental evaluations on the VisDrone, SeaDronesSeeV2, and UAVDT datasets demonstrate the effectiveness and plug-and-play capability of our proposed modules. Compared with the state-of-the-art YOLOv8 detector, the proposed EUAVDet achieves better performance in nearly all the metrics, including parameters, FLOPs, mAP, and FPS. The smallest version of EUAVDet (EUAVDet-n) contains only 1.34 M parameters and achieves over 20 fps on the Jetson Nano. Our algorithm strikes a better balance between detection accuracy and inference speed, making it suitable for edge-based UAV applications.

Hyperspectral images have been widely used for target detection due to their abundant spatial and spectral information. In this paper, a shadow-insensitive hyperspectral target detection framework based on exposure fusion is proposed, which consists of the following major steps. First, the input hyperspectral image is divided into two parts, namely the shadow region and the non-shadow region. Second, total variation-based feature extraction and over-exposure operation are performed on the input image to produce two feature images, i.e., the original feature image and the over-exposure image. Third, a self-guided constrained energy minimization (SGCEM) detector is performed on the two feature images to detect the targets in shadow and non-shadow regions, respectively. Finally, the detection results obtained on the original feature image and the over-exposure image are fused to acquire the final detection result. Extensive experiments conducted on real-world data illustrate that the proposed method can achieve satisfactory results when shadow exists.

Pansharpening has benefited from the development of deep learning (DL) and has achieved excellent results. However, most DL-based methods extract local features by convolutional neural network and do not integrate global features. Moreover, these methods only extract high frequency features on the high-pass domain or only consider image features on the intensity domain. The method that only considers features in one domain may result in insufficient extraction of spatial and spectral features. Therefore, we propose a dynamic local-global network model on dual domain, i.e., high-pass domain and intensity domain. The dynamic local-global feature extraction block (DLGB) is designed to dynamically integrate local and global features to improve the representation capability of the network. To decrease the computational complexity of global feature extraction, a lightweight biaxial nonlocal attention (BNLA) that captures global spatial features in horizontal and vertical directions is proposed. Experiments on GeoEye-1, QuickBird and WorldView-3 datasets show that the proposed method presents better fusion performance on objective evaluation indexes and subjective perception.

Single-modal image registration methods are generally not feasible for visible and infrared images. Besides, multi-modal image registration methods still suffer from uneven distribution of extracted features, low repeatability, and ambiguous features. To address these issues, a coarse-to-fine infrared and visible image registration approach for dual sensor UAV imaging system is proposed, which is resilient to the difference of focal lengths and field of view. First, in the coarse registration step, the infrared image is transformed to the same scale as the visible image by using the similarity transformation. This operation makes the proposed method robust to the variation of field of view. Then, the feature point pairs are initialized using feature detectors in the infrared image’s blocked phase congruency feature map. Next, the feature point pairs are optimized by estimating the offset based on the relationship between the constructed feature descriptors. Finally, using elastic deformation, the pixel-level registered infrared image is obtained. Extensive experiments demonstrate the superior performance of the proposed coarse-to-fine image registration methodology in the real infrared-visible image pairs. The code and dataset are available at https://drive.google.com/drive/folders/1mpUWwHUbKTrBdOrNMNRRnuJclDUAC7nU?usp=sharing.