UFPF: A Universal Feature Perception Framework for Microscopic Hyperspectral Images
- 5 August 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34 (10577149) , 5513-5526
Abstract
In recent years, deep learning has shown immense promise in advancing medical hyperspectral imaging diagnostics at the microscopic level. Despite this progress, most existing research models remain constrained to single-task or single-scene applications, lacking robust collaborative interpretation of microscopic hyperspectral features and spatial information, thereby failing to fully explore the clinical value of hyperspectral data. In this paper, we propose a microscopic hyperspectral universal feature perception framework (UFPF), which extracts high-quality spatial-spectral features of hyperspectral data, providing a robust feature foundation for downstream tasks. Specifically, this innovative framework captures different sequential spatial nearest-neighbor relationships through a hierarchical corner-to-center mamba structure. It incorporates the concept of “progressive focus towards the center”, starting by emphasizing edge information and gradually refining attention from the edges towards the center. This approach effectively integrates richer spatial-spectral information, boosting the model’s feature extraction capability. On this basis, a dual-path spatial-spectral joint perception module is developed to achieve the complementarity of spatial and spectral information and fully explore the potential patterns in the data. In addition, a Mamba-attention Mix-alignment is designed to enhance the optimized alignment of deep semantic features. The experimental results on multiple datasets have shown that this framework significantly improves classification and segmentation performance, supporting the clinical application of medical hyperspectral data. The code is available at: https://github.com/Qugeryolo/UFPF.Keywords
Funding Information
- National Key Research and Development Program of China (2024YFF0618201)
This publication has 32 references indexed in Scilit:
- Deform-Mamba Network for MRI Super-Resolution2024
- Swin-UMamba: Mamba-Based UNet with ImageNet-Based Pretraining2024
- Tri-Plane Mamba: Efficiently Adapting Segment Anything Model for 3D Medical Images2024
- Metasurface-empowered snapshot hyperspectral imaging with convex/deep (CODE) small-data learning theoryNature Communications, 2023
- A hyperspectral dataset of precancerous lesions in gastric cancer and benchmarks for pathological diagnosisJournal of Biophotonics, 2022
- Hyperspectral Imaging for Clinical ApplicationsBioChip Journal, 2022
- Enhancing hyperspectral imagingNature Machine Intelligence, 2021
- nnU-Net: a self-configuring method for deep learning-based biomedical image segmentationNature Methods, 2020
- ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed dataISPRS Journal of Photogrammetry and Remote Sensing, 2020
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015