S 2 TA-Fuse: Semantic-Superpixel Tokenized Attention for Spatial–Spectral Fusion
- 26 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 64 (01962892) , 1-14
Abstract
Superpixel paradigms have long been regarded as a well-established approach of mitigating spatial redundancy in hyperspectral images, yet their reliance on non-differentiable and irreversible segmentation renders them unsuitable for end-to-end spatial–spectral fusion (SSF). To tackle this limitation, this study introduces S²TA-Fuse, a transformer-based solver named Semantic-Superpixel Tokenized Attention for Fusion, which preserves the efficiency of superpixels while removing the need for explicit segmentation. The central design lies in a semantic attention mechanism that adaptively organizes pixels into deformable and content-aware semantic groups. Pixels sharing similar latent states are softly aggregated and encoded as compact tokens, upon which attention is computed to capture intricate long-range dependencies. This formulation endows the model with an inherent ability to accommodate scale variations while maintaining linear computational complexity with respect to the number of pixels. On top of the semantic backbone, two complementary components are devised. The Local Spectral Pyramid enhances the representation of multi-scale spectral cues in the spatial domain, whereas FreqNet supplements global information by modeling frequency-dependent variations through amplitude and phase decomposition. Comprehensive experiments on widely used benchmarks for spatial–spectral fusion demonstrate that S²TA-Fuse consistently surpasses the state of the art both in quantitative accuracy and visual fidelity.Keywords
Funding Information
- Natural Science Foundation of Zhejiang Province (LZ24F030012)
- Natural Science Foundation of Zhejiang Province (LMS25F020006)
- China Postdoctoral Science Foundation (2024M762911)
- National natural science foundation of China youth program (62506337)
- National Natural Science Foundation of China (62276232)
This publication has 58 references indexed in Scilit:
- Arbitrary-scale spatial-spectral fusion using kernel integral and progressive resamplingInformation Fusion, 2026
- Information-coupled MRI acceleration via multi-modal mapping and progressive maskingPattern Recognition, 2025
- Superpixel-aware credible dual-expert learning for land cover mapping using historical land cover productISPRS Journal of Photogrammetry and Remote Sensing, 2025
- Optical remote sensing image salient object detection via bidirectional cross-attention and attention restorationPattern Recognition, 2025
- Superpixel-based multi-scale multi-instance learning for hyperspectral image classificationPattern Recognition, 2024
- Diffusion model with disentangled modulations for sharpening multispectral and hyperspectral imagesInformation Fusion, 2023
- An Extensive Survey on Superpixel Segmentation: A Research PerspectiveArchives of Computational Methods in Engineering, 2023
- Multiscale spatial–spectral transformer network for hyperspectral and multispectral image fusionInformation Fusion, 2023
- Integrating remote sensing with ecology and evolution to advance biodiversity conservationNature Ecology & Evolution, 2022
- Remote sensing image fusion based on two-stream fusion networkInformation Fusion, 2020