Multifiltering MLP for Spectral Super-Resolution With Remote Sensing Image Verification
Open Access
- 26 August 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 17 (19391404) , 16646-16658
Abstract
Spectral super-resolution (SSR) has become an attractive approach to reconstructing hyperspectral images (HSIs) from more available RGB images or multi-spectral images (MSIs) owing to the powerful representation capability of deep learning. Prevailing efforts concentrate on reconstructing all channels of HSIs via constraining with groundtruth on the whole, ignoring precision of discriminative spectral characteristics (e.g., specific absorption peaks), which is critical for downstream tasks such as fine-grained hyperspectral classification to recognize analogous ground objects with similar spectral characteristics. Therefore, we introduce an efficient Multi-filtering MLP for SSR (MF-SSR) to reconstruct meticulous and high-fidelity HSIs in this paper, paving the roads toward downstream tasks based on recovered HSIs. A specific MF-MLP block is presented to individually reconstruct distinctive spectral-wise characteristics by repressing surrounding interferences from near channels. The flexible filtering ratios and positions in MF-MLP randomly disrupt the recovered main channel and surrounding channel range, which delineates specific absorption peaks to further represent the unique reflectivity or radiance of the light in each pixel. Besides, a Cross Spatio-Spectral Attention (CSSA) module is explicitly presented to simultaneously extract pixel-wise correlation and channel-wise affinity to amplify the consistency of the same substance and the diversities of different substances in a complementary mode. Comprehensive SSR experiments on four datasets and further classification verification based on reconstructed HSIs for Pavia University and GF-X datasets have demonstrated the superiority and practicability of our MF-SSR.Keywords
This publication has 54 references indexed in Scilit:
- Definition of spectrum by colors metamerism using images of plants by ordinary cameraOptics and Lasers in Engineering, 2024
- CMT: Convolutional Neural Networks Meet Vision Transformers2022
- HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging2022
- Multi-Stage Progressive Image Restoration2021
- Learning Enriched Features for Real Image Restoration and Enhancement2020
- Reconstructing Spectral Images from RGB-Images Using a Convolutional Neural Network2018
- HSCNN+: Advanced CNN-Based Hyperspectral Recovery from RGB Images2018
- NTIRE 2018 Challenge on Spectral Reconstruction from RGB Images2018
- 2D-3D CNN Based Architectures for Spectral Reconstruction from RGB Images2018
- Sparse Recovery of Hyperspectral Signal from Natural RGB Images2016