SSIF: Learning Continuous Image Representation for Spatial-Spectral Super-Resolution
Preprint
Abstract
Existing digital sensors capture images at fixed spatial and spectral resolutions (e.g., RGB, multispectral, and hyperspectral images), and each combination requires bespoke machine learning models. Neural Implicit Functions partially overcome the spatial resolution challenge by representing an image in a resolution-independent way. However, they still operate at fixed, pre-defined spectral resolutions. To address this challenge, we propose Spatial-Spectral Implicit Function (SSIF), a neural implicit model that represents an image as a function of both continuous pixel coordinates in the spatial domain and continuous wavelengths in the spectral domain. We empirically demonstrate the effectiveness of SSIF on two challenging spatio-spectral super-resolution benchmarks. We observe that SSIF consistently outperforms state-of-the-art baselines even when the baselines are allowed to train separate models at each spectral resolution. We show that SSIF generalizes well to both unseen spatial resolutions and spectral resolutions. Moreover, SSIF can generate high-resolution images that improve the performance of downstream tasks (e.g., land use classification) by 1.7%-7%.Keywords
All Related Versions
This publication has 30 references indexed in Scilit:
- ITSRN++: Stronger and better implicit transformer network for screen content image continuous super-resolutionDisplays, 2024
- Sphere2Vec: A general-purpose location representation learning over a spherical surface for large-scale geospatial predictionsISPRS Journal of Photogrammetry and Remote Sensing, 2023
- Implicit Neural Representations for Image Compression2022
- SphereSR: $360^{\circ}$ Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation2022
- Hyperspectral Image Super-Resolution with Deep Priors and Degradation Model Inversion2022
- A Spectral–Spatial Jointed Spectral Super-Resolution and Its Application to HJ-1A Satellite ImagesIEEE Geoscience and Remote Sensing Letters, 2021
- Single Image Super-Resolution via a Holistic Attention Network2020
- Mini-UAV-Borne Hyperspectral Remote Sensing: From Observation and Processing to ApplicationsIEEE Geoscience and Remote Sensing Magazine, 2018
- Hyperspectral Remote Sensing Data Analysis and Future ChallengesIEEE Geoscience and Remote Sensing Magazine, 2013
- On piecewise-linear basis functions and piecewise-linear signal expansionsIEEE Transactions on Acoustics, Speech, and Signal Processing, 1974