Explore Better Network Framework for High-Resolution Optical and SAR Image Matching
- 10 November 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 60 (01962892) , 1-18
Abstract
To fully explore the complementary information from optical and synthetic aperture radar (SAR) imageries, they need firstly to be co-registered with high accuracy. Due to the vast radiometric and geometric disparity, the problem to match high resolution optical and SAR images is quite challenging. The present deep learning based methods have shown advantages over the traditional approaches, but the performance increment is not significant. In this article, we explore better network framework for high resolution optical and SAR image matching from three aspects. Firstly, we propose an effective multilevel feature fusion method which helps to take advantage of both the low-level fine-grained features for precious feature location and the high-level semantic features for better discriminative ability. Secondly, a feature channel excitation procedure is conducted using a novel multi-frequency channel attention module, which is able to make image features of different types and multiple levels effectively collaborate with each other, and produce image matching features with high diversity. Thirdly, the self-adaptive weighting loss is introduced, with which, each sample is assigned with an adaptive weighting factor, and therefore information buried in all nearby samples can be better exploited. Under a pseudo-Siamese architecture, the proposed optical and SAR image matching network (OSMNet) is trained and tested on a large and diverse high resolution optical and SAR dataset. Extensive experiments demonstrate that each component of the proposed deep framework helps to improve the matching accuracy. Also, the OSMNet shows overwhelming superior to the state-of-the-art handcrafted approaches on imageries of different land-cover types.Keywords
This publication has 49 references indexed in Scilit:
- Statistical power of intensity- and feature-based similarity measures for registration of multimodal remote sensing images2016
- Gated Siamese Convolutional Neural Network Architecture for Human Re-identification2016
- Deep Metric Learning Using Triplet Network2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- MIND: Modality independent neighbourhood descriptor for multi-modal deformable registrationMedical Image Analysis, 2012
- SIFT Flow: Dense Correspondence across Scenes and Its ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- Modifications in the SIFT operator for effective SAR image matchingInternational Journal of Image and Data Fusion, 2010
- Estimation of Mutual Information: A Survey2009
- Histograms of Oriented Gradients for Human Detection2005
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004