Cross-Sensor Remote-Sensing Images Scene Understanding Based on Transfer Learning Between Heterogeneous Networks
- 14 October 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 19 (1545598X) , 1-5
Abstract
Over the past decades, the successful invention and employment of multiple sensors have marked the advent of a new era in multisensor remote-sensing (RS) images acquisition. To effectively utilize the massive multisensor images for RS scene understanding, we expect that a scene classification model learned with particular sensor data can generalize well to other sensor data. However, this is a very challenging task due to the cross-sensor data differences. In the deep learning (DL) pipeline, a common way to handle this challenging task is to fine-tune the models pretrained on source sensor data with limited labeled data from the target sensor. Unfortunately, fine-tune technique is usually applied between homogeneous networks, which may not be the best choice if the source and target data are largely different. To address these issues, we formulate the cross-sensor RS scene understanding problem as a heterogeneous network-oriented transfer learning problem, in which the source and the target networks are different and data-oriented selected. Afterward, the knowledge between heterogeneous networks is transferred using the pseudo-label recursive propagation mechanism inspired by the concept of knowledge distillation. To the best of our knowledge, this is the first time to investigate the cross-sensor scene classification problem by constructing such a heterogeneous networks’ transfer scheme in RS fields. Our experiments using two cross-sensor RS datasets [aerial images $\rightarrow $ multispectral images (MSIs) and aerial images $\rightarrow $ hyper-spectral images (HSIs)] demonstrated that the proposed transfer learning strategy based on heterogeneous networks outperforms the supervised learning (SL) and fine-tune scheme for cross-sensor scene classification.
Keywords
Funding Information
- High Performance Computing Center of Central South University
- National Natural Science Foundation of China (42171376)
- National Natural Science Foundation of China (41771458)
- National Natural Science Foundation of China (41871364)
- Natural Science Foundation of Hunan (2021JJ30815)
- Young Elite Scientists Sponsorship Program by Hunan Province of China (2018RS3012)
- Hunan Science and Technology Department Innovation Platform Open Fund Project (18K005)
- Fundamental Research Funds for the Central Universities of Central South University (2021zzts0821)
This publication has 19 references indexed in Scilit:
- Convolutional Neural Network for Remote-Sensing Scene Classification: Transfer Learning AnalysisRemote Sensing, 2019
- Further Exploring Convolutional Neural Networks’ Potential for Land-Use Scene ClassificationIEEE Geoscience and Remote Sensing Letters, 2019
- Remote Sensing Satellites for Digital Earth2019
- EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover ClassificationIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019
- Scene Classification With Recurrent Attention of VHR Remote Sensing ImagesIEEE Transactions on Geoscience and Remote Sensing, 2018
- Remote Sensing Image Scene Classification: Benchmark and State of the ArtProceedings of the IEEE, 2017
- Bag-of-visual-words and spatial extensions for land-use classification2010
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Multiresolution gray-scale and rotation invariant texture classification with local binary patternsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Color indexingInternational Journal of Computer Vision, 1991