Build General Pansharpening Model Through a Simple Sensor-Specific Model Selection Strategy
- 17 November 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 9108-9113
Abstract
Deep-learning-based pansharpening methods have become a hot field in the remote sensing community. However, most existing deep learning pansharpening methods are conducted in a sensor-specific and supervised way. That is, pairs of panchromatic (Pan) and multispectral (MS) images acquired from a given sensor are fed into networks. The trained model can achieve a satisfactory fusion result only if the images are fused from the same sensor as the training images. The trained model from one sensor cannot be transferred to another. How to train a general model suitable for different sensors is a challenging problem. In this context, we propose a simple sensor-type detecting strategy, which can be integrated with any sensor-specific pansharpening methods and result in a general fusion framework across different sensors. More specifically, for a given deep-learning-based pansharpening method, sufficient Pan and MS image pairs from multiple sensors are fed into the network respectively, and models from multiple sensors are obtained to construct a model pool. Thereafter, a pair of local patches of MS and Pan from the test images are fed into existing sensor-specific models in the pool respectively. The pansharpening quality of outputs are evaluated with multiple fusing indices, such as quality without reference index (QNR). The model that achieves the best performance in terms of index is selected as the candidate model and further employs the images over the whole scene. To evaluate the effectiveness of the proposed framework, WSDFNet, a state-of-the-art pansharpening method was trained with GaoFen-1, QuickBird, and WorldView-3 datasets respectively. The QNR index is the optimal index for predicting the sensor type. With 100 images from three datasets as testing images, the correct accuracy rate of the statistical QNR index on sensor differentiation performance is 86, 69, and 62, respectively, which indicates this model is promising for distinguishing the sensor type. This proposed prediction strategy is simple and flexible, and it can be combined with existing deep-learning methods to form a complete fusion process without distinguishing the sensor type.Keywords
This publication has 20 references indexed in Scilit:
- Assessing Full-Resolution Pansharpening Quality: A Comparative Study of Methods and MeasurementsIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023
- Weighted Shallow-Deep Feature Fusion Network for Pansharpening2021
- A Benchmarking Protocol for Pansharpening: Dataset, Preprocessing, and Quality AssessmentIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021
- A Comprehensive Study on Computational Pansharpening Techniques for Remote Sensing ImagesArchives of Computational Methods in Engineering, 2021
- A review of image fusion techniques for pan-sharpening of high-resolution satellite imageryISPRS Journal of Photogrammetry and Remote Sensing, 2021
- Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challengesInformation Fusion, 2019
- PanNet: A Deep Network Architecture for Pan-Sharpening2017
- Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual NetworkIEEE Geoscience and Remote Sensing Letters, 2017
- Deep Residual Learning for Image Recognition2016
- Pan-sharpening of multispectral images: a critical review and comparison2004