Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing Images
Open Access
- 16 May 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 16 (19391404) , 4739-4754
Abstract
Deep convolutional networks are powerful for local feature learning and have shown advantages in image matching and registration. However, the significant differences between cross-modal images increase the challenge of image registration. The deep network should extract modality-invariant features to identify the matching samples and discriminative features to separate the non-matching samples. The deep network can extract features invariant to the image modality changes by multiple non-linear mapping layers. However, it does not inevitably lose rich details and affect the discrimination of features, degrading registration performances. This paper proposes a novel deep wavelet learning network (DW-Net) for local feature learning. It incorporates spectral information into deep convolutional features for improving cross-modal image matching and registration. Specifically, this paper aims to learn the multi-resolution wavelet features through multi-level wavelet transform (WT) and the convolutional network. The cross-modal images are divided into low-frequency and high-frequency parts through WT. DW-Net can adaptively extract the shared features from the low-frequency part and useful details from the high-frequency part, which can enhance the modality-invariance and discrimination of features. Additionally, the multi-resolution wavelet features contain multi-scale information and contribute to improving the matching accuracy. Extensive experiments demonstrate the significant advantages in terms of the accuracy and robustness of DW-Net on cross-modal remote sensing image registration. DW-Net can increase the image patch matching accuracy by 3.7% and improve image registration probability by 12.1%. Moreover, DW-Net shows strong generalization performances from low-resolution to high-resolution and from optical-SAR to other cross-modal image registration.Keywords
Funding Information
- National Natural Science Foundation of China (62201407, 62271377, 61977052)
- Key Research and Development Program of Shannxi (2021ZDLGY0106, 2022ZDLGY0112)
- National Key R&D Program of China (2021ZD0110404)
- Key Scientific Technological Innovation Research Project by Ministry of Education
- National Key R&D Program of China (SQ2021AAA010888, 2021ZD0110404)
- China Postdoctoral Science Foundation (2022M722496)
- Fundamental Research Funds for the Central Universities (QTZX23020)
This publication has 57 references indexed in Scilit:
- Discriminative Learning of Deep Convolutional Feature Point Descriptors2015
- Learning to compare image patches via convolutional neural networks2015
- A local descriptor based registration method for multispectral remote sensing images with non-linear intensity differencesISPRS Journal of Photogrammetry and Remote Sensing, 2014
- Change Detection in Synthetic Aperture Radar Images based on Image Fusion and Fuzzy ClusteringIEEE Transactions on Image Processing, 2011
- ASIFT: An Algorithm for Fully Affine Invariant ComparisonImage Processing On Line, 2011
- Modifications in the SIFT operator for effective SAR image matchingInternational Journal of Image and Data Fusion, 2010
- Speeded-Up Robust Features (SURF)Computer Vision and Image Understanding, 2008
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Multimodality image registration by maximization of mutual informationIEEE Transactions on Medical Imaging, 1997
- Random sample consensusCommunications of the ACM, 1981