Tong Zhou

Hyperspectral imagery (HSI), characterized by the integration of both spatial and spectral information, is widely employed in various fields, such as environmental monitoring, geological exploration, precision agriculture, and medical imaging. Hyperspectral image classification (HSIC), as a key research direction, aims to establish a mapping relationship between pixels and land-cover categories. Nevertheless, several challenges persist, including difficulties in feature extraction, the trade-off between effective integration of local and global features, and spectral redundancy. We propose FreqMamba, a novel model that efficiently combines CNN, a custom attention mechanism, and the Mamba architecture. The proposed framework comprises three key components: (1) A novel multi-scale deformable convolution feature extraction module equipped with spectral attention, which processes spectral and spatial information through a dual-branch structure to enhance feature representation for irregular terrain contours; (2) a novel group-separated attention module that integrates group convolution with group-separated self-attention, effectively balancing local feature extraction and global contextual modeling; (3) a newly introduced bidirectional scanning Mamba branch that efficiently captures long-range dependencies with linear computational complexity. The proposed method achieves optimal performance on multiple benchmark datasets, including QUH-Tangdaowan, QUH-Qingyun, and QUH-Pingan, with the highest overall accuracy reaching 97.47%, average accuracy reaching 93.52%, and a Kappa coefficient of 96.22%. It significantly outperforms existing CNN, Transformer, and SSM-based methods, demonstrating its effectiveness, robustness, and superior generalization capability.

Hyperspectral images, characterized by rich spectral information, enable precise pixel-level classification and are thus widely employed in remote sensing applications. Although convolutional neural networks (CNNs) have demonstrated effectiveness in hyperspectral image processing, their limited receptive fields constrain their capacity to capture long-range dependencies. Transformers excel at modeling long-range features for hyperspectral image classification (HSIC). Yet, they often overlook effective representation of local spectral–spatial characteristics while incurring computational redundancy from numerous classification-irrelevant tokens. To address these challenges, we propose EDTST, a state-of-the-art Vision Transformer architecture specifically designed for efficient hyperspectral image classification. The model utilizes a large-kernel 3D convolution block to extract deep spectral–spatial features. A 2D convolution block further refines these features, followed by a novel attention mechanism with dynamic token pruning that substantially reduces the computational load by focusing on the most pertinent features. The process concludes with an adaptive average pooling layer and a fully connected layer for classification. Extensive experiments on four standard hyperspectral datasets demonstrate that EDTST achieves the highest classification accuracy, with a notable 3% improvement in overall accuracy on the WHU-Hi-HanChuan dataset, while requiring the shortest training and inference time among all compared state-of-the-art models from the past three years. These results validate the efficacy of our approach in achieving superior performance with markedly improved computational efficiency.

Vision Transformers have demonstrated remarkable performance in hyperspectral image classification tasks. However, their complex computational mechanisms and excessive parameterization severely restrict deployment on resource-constrained platforms such as FPGAs and embedded CPUs. As a key technology for lightweight deep models, binary quantization achieves significant parameter compression and computational acceleration by binarizing activations and weights. However, binary quantization in Transformers faces challenges like degradation of feature representation capability after binarizing self-attention mechanisms and decline in fusion efficiency of multi-scale spectral-spatial information, leading to relatively lagging progress in this field. To address these issues, this study proposes a novel binary Vision Transformer architecture tailored for hyperspectral image classification. Built upon traditional Transformers, the approach innovatively introduces a self-adaptive softmax binarization module, which dynamically adjusts the binarization threshold distribution to effectively mitigate discretization errors in gradient propagation during the binarization process. Meanwhile, a multi-branch average pooling block is designed to enable hierarchical aggregation of features across different spectral dimensions, significantly enhancing the model's ability to represent complex spectral-spatial correlations in hyperspectral data. The proposed method achieves over 99% parameter binarization and reduces floating-point computation by more than 89% compared to full-precision counterparts, while maintaining competitive classification accuracy. Experiments conducted on seven benchmark hyperspectral datasets demonstrate the effectiveness of our approach in balancing computational efficiency and classification performance.

Automatic modulation recognition plays a critical role in military applications, particularly in electronic warfare, spectrum surveillance, and secure communication systems. The precise identification of signal modulation modes is crucial for ensuring the efficacy, security, and efficiency of communications. Given the problems of limited feature extraction capability and performance degradation when dealing with low signal-to-noise ratio (SNR) signals, this work proposes a novel model architecture that combines the Transformer with relative position encoding and separable convolution (SC) in the complex domain. The network can directly process the complex representation of signals to capture features in the time-frequency domain and enhance the ability to recognize complex signals. This method introduces relative position encoding in the complex domain into the Transformer framework, which uses complex attention mechanisms and adaptive position encoding to enhance the model’s long-range modeling capability. At the same time, it improves computational efficiency and local feature extraction capability by introducing SC layers. Then, we construct an attention-driven feature fusion module, which can automatically adjust the weight ratio between features to achieve the optimal combination of features. The model shows excellent classification performance under various SNR conditions in RadioML2016.10a and RadioML2016.10b datasets, especially in low SNR environments, which is significantly improved compared to other methods. The research not only provides a new solution for AMR tasks but also expands new ideas for applying the Transformer model in signal processing.

Angle of arrival (AOA) estimation and instantaneous frequency measurement (IFM) are important aspects of radar, communication, and electronic warfare, and there is an urgent need to find intelligent solutions that are faster and more accurate than traditional methods. Currently, deep learning (DL) has increasingly strong learning and feature extraction capabilities, which brings new opportunities to the field of electromagnetic signal processing. However, the models in DL have a large number of parameters, high storage requirements, and long computational latency, and not practical for deploying applications on resource-limited devices. In this paper, we build a complete AOA and IFM system for the first time, taking advantage of microwave photonics (MWP) technology, DL, and features of field programmable gate array (FPGA). Then, a novel hardware-friendly pruning-quantization-iteration compression (PQIC) algorithm is proposed for the post-processing of MWP signal measurement applications. By reducing the data width from 32 bits to 4 bits, the compressed algorithm reduces parameter storage requirements and hardware implementation complexity with negligible performance loss. Finally, we deploy the acceleration system and test the actual collected signal data on FPGA while running 200 MHz. The results show that with a 14× model compression and a 3.99× reduction in operations, the accuracy of AOA reaches 98.82%, with the mean absolute error (MAE) of 0.27°. More importantly, the running latency is only 20.78 μs, meeting the real-time processing requirements of MWP signal processing. It is also applicable to real-world signal intelligence processing and demonstrates superior performance compared to other existing algorithms in this field.

Deep learning techniques have been widely investigated as an effective method for signal measurement in recent years. However, most existing deep learning-based methods still face difficulty in deploying on embedded platforms and perform poorly in real-time applications. To address this, this paper develops two accelerators, as the core of the signal measurement system, for intelligent signal processing. Firstly, by introducing the idea of automated framework, we propose a simplest deep neural network (DNN)-based hardware structure, which automatically maps algorithms to hardware modules, supports configurable parameters, and has the advantage of low latency, with an average inference time of only 3.5 μs. Subsequently, another accelerator is designed with the efficient hardware structure of the long short-term memory (LSTM) + DNN model, demonstrating outstanding performance with a classification accuracy of 98.82%, mean absolute error (MAE) of 0.27°, and root mean square errors (RMSE) of 0.392° after model compression. Moreover, parallel optimization strategies are exploited to further reduce latency and support simultaneous frequency and direction measurement tasks. Finally, we test the actual collected signal data on the XCVU13P field programmable gate array (FPGA). The results show that the time of inference saves 28–31% for the DNN model and 71–73% for the LSTM + DNN model compared to running on graphic processing unit (GPU). In addition, the parallel strategies further decrease the delay by 23.9% and 37.5% when processing continuous data. The FPGA-based and deep learning-assisted hardware accelerators significantly improve real-time performance and provide a promising solution for signal measurement.

The fusion of hyperspectral and multispectral images is an important strategy for enhancing the spatial resolution of hyperspectral images. With the rapid advancement of multispectral imaging technology, the disparity in spatial resolution between multispectral and hyperspectral images is increasing. In certain scenarios, termed high-magnification, this difference can exceed 32×. Previous methods do not perform well under high-magnification fusion, and naturally, a challenge arises in achieving effective high-magnification super-resolution fusion. In light of the above analysis, this paper introduces a novel algorithm for high-magnification super-resolution fusion of hyperspectral and multispectral images based on the joint optimization of spatial and spectral information. Specifically, our algorithm consists of three stages: 1) a fast preliminary fusion stage based on the Moore-Penrose inverse and singular value correlation priors for the rapid acquisition of preliminary solutions, 2) a joint spatial-spectral optimization stage where a coupled optimization framework is constructed to achieve integrated optimization of spatial and spectral information, and 3) an error backpropagation optimization stage where an effective error optimization term is introduced to further refine the fusion performance. We conducted extensive experiments on widely employed publicly available simulated datasets and real datasets. The experimental results unequivocally indicate that our proposed methodology consistently exhibits superior fusion performance compared with state-of-the-art methods, even under the condition of ≥60× super-resolution.

As human life continues improving, applications like virtual reality (VR) and augmented reality (AR) necessitate increasingly higher-quality 3D reconstructions. With the advancements in neural implicit 3D surface and volume rendering, multi-view 3D reconstruction has garnered significant attention. A prevailing issue in this domain is that the direct combination of neural implicit surface and volume rendering typically considers only photometric consistency loss, leading to an under-constrained surface problem. To address this problem, we develop a double-supervised approach and an integrated network for implicit surface and volume rendering, enabling the generation of a 3D surface model of an object from a set of multi-view images. In our approach, the object surface is represented by a signed distance function, and a 3D surface reconstruction model is jointly trained with photometric consistency constraints and geometry constraints. Experiments demonstrate that by enhancing prior geometry supervision and integrating the double-supervised network architecture into neural implicit 3D reconstruction, we can achieve accurate and high-quality 3D reconstruction.