Band-Wise Hyperspectral Image Pansharpening Using CNN Model Propagation
Open Access
- 4 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-18
Abstract
Hyperspectral (HS) pansharpening has received a growing interest in the last few years as testified by a large number of research papers and challenges. It consists in a pixel-level fusion between a lower resolution HS datacube and a higher resolution single-band image, the panchromatic (PAN) image, with the goal of providing an HS datacube at PAN resolution. Due to their powerful representational capabilities, deep learning models have succeeded to provide unprecedented results on many general-purpose image processing tasks. However, when moving to domain-specific problems, as in this case, the advantages with respect to traditional model-based approaches are much lesser clear-cut due to several contextual reasons. Scarcity of training data, lack of ground truth (GT), and data shape variability are some such factors that limit the generalization capacity of the state-of-the-art deep learning networks for HS pansharpening. To cope with these limitations, in this work, we propose a new deep learning method, which inherits a simple single-band unsupervised pansharpening model nested in a sequential band-wise adaptive scheme, where each band is pansharpened refining the model tuned on the preceding one. By doing so, a simple model is propagated along the wavelength dimension, adaptively and flexibly, with no need to have a fixed number of spectral bands, and, with no need to dispose of large, expensive, and labeled training datasets. The proposed method achieves very good results on our datasets, outperforming both traditional and deep learning reference methods. The implementation of the proposed method can be found at https://github.com/giu-guarino/R-PNN .Keywords
All Related Versions
This publication has 69 references indexed in Scilit:
- Hyperspectral and Multispectral Image Fusion Based on a Sparse RepresentationIEEE Transactions on Geoscience and Remote Sensing, 2015
- A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based RegularizationIEEE Transactions on Geoscience and Remote Sensing, 2014
- A Critical Comparison Among Pansharpening AlgorithmsIEEE Transactions on Geoscience and Remote Sensing, 2014
- A variational approach to hyperspectral image fusion2009
- Multispectral and Panchromatic Data Fusion Assessment Without ReferencePhotogrammetric Engineering & Remote Sensing, 2008
- Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan DataIEEE Transactions on Geoscience and Remote Sensing, 2007
- MTF-tailored Multiscale Fusion of High-resolution MS and Pan ImageryPhotogrammetric Engineering & Remote Sensing, 2006
- Sharpening of very high resolution images with spectral distortion minimization2003
- Binary partition tree as an efficient representation for image processing, segmentation, and information retrievalIEEE Transactions on Image Processing, 2000
- A wavelet transform method to merge Landsat TM and SPOT panchromatic dataInternational Journal of Remote Sensing, 1998