Hyperspectral and LiDAR Classification via Frequency Domain-Based Network
- 30 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-17
Abstract
Local-global feature learning method based on deep learning has significantly improved the collaborative classification of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data. However, HSI encompasses numerous bands with significant inter-band correlations. Addressing how to efficiently capture spatial, spectral, and elevation information from hyperspectral and LiDAR data while considering data redundancy and enhancing feature representation of land cover will contribute to enhancing classification effectiveness. Combining frequency feature learning method with convolutional neural networks (CNNs), transformers, and other architectures to construct an end-to-end feature learning network framework is an effective method. Therefore, this paper proposes a frequency domain-based network (FDNet) for the classification of HSI and LiDAR data using a frequency local feature learning framework and a self-attention mechanism based on fast fourier transform (FFT). FDNet could effectively capture local efficient frequency features of spatial, spectral, and elevation information in HSI and LiDAR data in an adaptive feature learning style, and embedding convolutional offsets into the frequency domain-based transformer network not only enhances local features but also effectively captures global semantic characteristics of land covers while reducing computational complexity. We validated the efficacy of FDNet across three publicly available datasets and a particularly challenging self-constructed dataset, denoted as Yancheng dataset. The source codes will be available at https://github.com/RSIPNJUPT/ FDNet.Keywords
Funding Information
- National Natural Science Foundation of China (62101280)
- Natural Science Foundation of Jiangsu Province (BK20210588)
- China Postdoctoral Science Foundation (2023M731781)
- Open Foundations of Jiangsu Province Engineering Research Center of Airborne Detecting and Intelligent Perceptive Technology (JSECF2023-01, JSECF2023-05, JSECF2023-08)
- Key Laboratory of Radar Imaging and Microwave Photonics, Nanjing University of Aeronautics and Astronautics, Ministry of Education (NJ20230005)
- Jiangsu Geological Bureau Research Project (2021KY14, 2023KY11)
- Nanjing University of Posts and Telecommunications Science Foundation (NY222107)
This publication has 63 references indexed in Scilit:
- Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring2023
- Learning A Sparse Transformer Network for Effective Image Deraining2023
- SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer2023
- KVT: k-NN Attention for Boosting Vision Transformers2022
- AdaViT: Adaptive Vision Transformers for Efficient Image Recognition2022
- Restormer: Efficient Transformer for High-Resolution Image Restoration2022
- NFormer: Robust Person Re-identification with Neighbor Transformer2022
- Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning modelISPRS Journal of Photogrammetry and Remote Sensing, 2021
- Wavelet Integrated CNNs for Noise-Robust Image Classification2020
- FusAtNet: Dual Attention based SpectroSpatial Multimodal Fusion Network for Hyperspectral and LiDAR Classification2020