Longhao Wang

Rapidly stitching unmanned aerial vehicle (UAV) imagery to produce high-resolution fast-stitch maps is key to UAV emergency mapping. However, common problems such as gaps and ghosting in image stitching remain challenging and directly affect the visual interpretation value of the imagery product. Inspired by the data characteristics of high-precision satellite images with rich access and geographic coordinates, a seamless stitching method is proposed for emergency response without the support of ground control points (CGPs) and global navigation satellite systems (GNSS). This method aims to eliminate stitching traces and solve the problem of stitching error accumulation. Firstly, satellite images are introduced to support image alignment and geographic coordinate acquisition simultaneously using matching relationships. Then a dynamic contour point set is constructed to locate the stitching region and adaptively extract the fused region of interest (FROI). Finally, the gradient weight cost map of the FROI image is computed and the Laplacian pyramid fusion rule is improved to achieve seamless production of the fast-stitch image map with geolocation information. Experimental results indicate that the method is well adapted to two representative sets of UAV images. Compared with the Laplacian pyramid fusion algorithm, the peak signal-to-noise ratio (PSNR) of the image stitching results can be improved by 31.73% on average, and the mutual information (MI) can be improved by 19.98% on average. With no reliance on CGPs or GNSS support, fast-stitch image maps are more robust in harsh environments, making them ideal for emergency mapping and security applications.

Three-dimensional (3D) scene reconstruction plays an important role in digital cities, virtual reality, and simultaneous localization and mapping (SLAM). In contrast to perspective images, a single panoramic image can contain the complete scene information because of the wide field of view. The extraction and matching of image feature points is a critical and difficult part of 3D scene reconstruction using panoramic images. We attempted to solve this problem using convolutional neural networks (CNNs). Compared with traditional feature extraction and matching algorithms, the SuperPoint (SP) and SuperGlue (SG) algorithms have advantages for handling images with distortions. However, the rich content of panoramic images leads to a significant disadvantage of these algorithms with regard to time loss. To address this problem, we introduce the Improved Cube Projection Model: First, the panoramic image is projected into split-frame perspective images with significant overlap in six directions. Second, the SP and SG algorithms are used to process the six split-frame images in parallel for feature extraction and matching. Finally, matching points are mapped back to the panoramic image through coordinate inverse mapping. Experimental results in multiple environments indicated that the algorithm can not only guarantee the number of feature points extracted and the accuracy of feature point extraction but can also significantly reduce the computation time compared to other commonly used algorithms.

Heterogeneous images acquired from various platforms and sensors provide complementary information. However, to use that information in applications such as image fusion and change detection, accurate image matching is essential to further process and analyze these heterogeneous images, especially if they have significant differences in radiation and geometric characteristics. Therefore, matching heterogeneous remote sensing images is challenging. To address this issue, we propose a feature point matching method named Cross and Self Attentional Matcher (CSAM) based on Attention mechanisms (algorithms) that have been extensively used in various computer vision-based applications. Specifically, CSAM alternatively uses self-Attention and cross-Attention on the two matching images to exploit feature point location and context information. Then, the feature descriptor is further aggregated to assist CSAM in creating matching point pairs while removing the false matching points. To further improve the training efficiency of CSAM, this paper establishes a new training dataset of heterogeneous images, including 1,000,000 generated image pairs. Extensive experiments indicate that CSAM outperforms the existing feature extraction and matching methods, including SIFT, RIFT, CFOG, NNDR, FSC, GMS, OA-Net, and Superglue, attaining an average precision and processing time of 81.29% and 0.13 s. In addition to higher matching performance and computational efficiency, CSAM has better generalization ability for multimodal image matching and registration tasks.

Image matching is a key research issue in the intelligent processing of remote sensing images. Due to the large phase differences or apparent differences in ground features between unmanned aerial vehicle imagery and satellite imagery, as well as the large number of sparsely textured areas, image matching between the two types of imagery is very difficult. Tackling the difficult problem of matching unmanned aerial vehicle imagery and satellite imagery, a feature sparse region detection and matching enhancement algorithm (SD-ME) is proposed in this study. First, the SuperGlue algorithm was used to initially match the two images, and feature-sparse region detection was performed with the help of the image features and initial matching results, with the detected feature sparse areas stored in a linked list one by one. Then, according to the order of storage, feature re-extraction was performed on the feature-sparse areas individually, and an adaptive threshold feature screening algorithm was proposed to filter and screen the re-extracted features. This retains only high-confidence features in the region and improves the reliability of matching enhancement results. Finally, local features with high scores that were re-extracted in the feature-sparse areas were aggregated and input to the SuperGlue network for matching, and thus, reliable matching enhancement results were obtained. The experiment selected four pairs of un-manned aerial vehicle imagery and satellite imagery that were difficult to match and compared the SD-ME algorithm with the SIFT, ContextDesc, and SuperGlue algorithms. The results revealed that the SD-ME algorithm was far superior to other algorithms in terms of the number of correct matching points, the accuracy of matching points, and the uniformity of distribution of matching points. The number of correctly matched points in each image pair increased by an average of 95.52% compared to SuperGlue. The SD-ME algorithm can effectively improve the matching quality between unmanned aerial vehicle imagery and satellite imagery and has practical value in the fields of image registration and change detection.