Decoupled-and-Coupled Networks: Self-Supervised Hyperspectral Image Super-Resolution With Subpixel Fusion
Top Cited Papers
- 13 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-12
Abstract
Enormous efforts have been recently made to super-resolve hyperspectral (HS) images with the aid of high spatial resolution multispectral (MS) images. Most prior works usually perform the fusion task by means of multifarious pixel-level priors. Yet the intrinsic effects of a large distribution gap between HS-MS data due to differences in the spatial and spectral resolution are less investigated. The gap might be caused by unknown sensor-specific properties or highly-mixed spectral information within one pixel (due to low spatial resolution). To this end, we propose a subpixel-level HS super-resolution framework by devising a novel decoupled-and-coupled network, called DC-Net, to progressively fuse HS-MS information from the pixel- to subpixel-level, from the image- to feature-level. As the name suggests, DC-Net first decouples the input into common (or cross-sensor) and sensor-specific components to eliminate the gap between HS-MS images before further fusion, and then thoroughly blends them by a model-guided coupled spectral unmixing (CSU) net. More significantly, we append a self-supervised learning module behind the CSU net by guaranteeing material consistency to enhance the detailed appearance of the restored HS product. Extensive experimental results show the superiority of our method both visually and quantitatively and achieve a significant improvement in comparison with the state-of-the-art.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2022YFB3903401)
- National Natural Science Foundation of China (42241109, 42271350, 62201553)
- MIAI@Grenoble Alpes (ANR-19-P3IA-0003)
- AXA Research Fund
This publication has 43 references indexed in Scilit:
- Hyperspectral Image Super-Resolution via Non-Negative Structured Sparse RepresentationIEEE Transactions on Image Processing, 2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Hyperspectral Super-Resolution by Coupled Spectral Unmixing2015
- Hyperspectral Pansharpening: A ReviewIEEE Geoscience and Remote Sensing Magazine, 2015
- Bayesian sparse representation for hyperspectral image super resolution2015
- The Spectral Response of the Landsat-8 Operational Land ImagerRemote Sensing, 2014
- Hyperspectral Target Detection : An Overview of Current and Future ChallengesIEEE Signal Processing Magazine, 2013
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- Spectral unmixingIEEE Signal Processing Magazine, 2002
- The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer dataRemote Sensing of Environment, 1993