An evaluation of conventional and deep learning‐based image‐matching methods on diverse datasets
- 25 April 2023
- journal article
- research article
- Published by Wiley in The Photogrammetric Record
- Vol. 38 (182) , 137-159
Abstract
Image matching plays an important role in photogrammetry, computer vision and remote sensing. Modern deep learning‐based methods have been proposed for image matching; however, whether they will surpass and take the place of the conventional handcrafted methods in the remote sensing field still remains unclear. A comprehensive evaluation on stereo remote sensing images is also lacking. This paper comprehensively evaluates the performance of conventional and deep learning‐based image‐matching methods by dividing the matching process into feature point extraction, description and similarity measure on various datasets, including images captured from close‐range indoor and outdoor scenarios, unmanned aerial vehicles (UAVs) and satellite platforms. Different combinations of the three steps are evaluated. The experimental results reveal that, first, the performance of the different combinations varies between individual datasets, and it is difficult to determine the best combination. Second, by using more comprehensive indicators on all of the datasets, that is, the average rank and absolute rank, the combination of scale‐invariant feature transform (SIFT), ContextDesc and the nearest neighbour distance ratio (NNDR), and also the original SIFT, achieve the best results, and are recommended for use in remote sensing. Third, the deep learning‐based Sub‐SuperPoint extractor obtains a good performance, and is second only to SIFT. The learning based ContextDesc descriptor is as effective as the SIFT descriptor, and the learning based SuperGlue matcher is not as stable as NNDR, but leads to a few top‐performing combinations. Finally, the handcrafted methods are generally faster than the deep learning‐based methods, but the efficiency of the latter is acceptable. We conclude that although a full deep learning‐based method/combination has not yet beaten the conventional methods, there is still much room for improvement with the deep learning‐based methods because large‐scale aerial and satellite training datasets remain to be constructed, and specific methods for remote sensing images remain to be developed. The performance of the different combinations of feature extractor, descriptor and similarity measure varies between individual datasets. The combination of SIFT, ContextDesc and NNDR, and also the original SIFT, achieve the best results when using more comprehensive indicators on all the datasets. For extractor, the learning based Sub‐SuperPoint is second only to SIFT; for descriptor, learning‐based ContextDesc is as effective as the SIFT descriptor; and for matcher, learning‐based SuperGlue is not as stable as NNDR.Keywords
Funding Information
- National Key Research and Development Program of China (2019YFE0125300)
- National Natural Science Foundation of China (42171430)
This publication has 45 references indexed in Scilit:
- Domain-size pooling in local descriptors: DSP-SIFT2015
- Comparative Evaluation of Binary Features2012
- BRIEF: Binary Robust Independent Elementary Features2010
- Adaptive and Generic Corner Detection Based on the Accelerated Segment Test2010
- ASIFT: A New Framework for Fully Affine Invariant Image ComparisonSIAM Journal on Imaging Sciences, 2009
- Speeded-Up Robust Features (SURF)Computer Vision and Image Understanding, 2008
- CSIFT: A SIFT Descriptor with Color Invariant Characteristics2006
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- A Combined Corner and Edge Detector1988
- Induction of decision treesMachine Learning, 1986