Wenfang Sun

Hyperspectral imaging (HSI) is an important imaging modality in the field of optical remote sensing, capable of simultaneously capturing spatial structures and continuous spectral responses related to material properties. Therefore, compared with conventional red-green-blue (RGB) images, it can provide richer representations for understanding dynamic scenes. However, most existing video object segmentation methods have been developed for RGB videos, while high-quality hyperspectral video datasets with frame-by-frame pixel-level annotations remain scarce. To address this gap, this study constructed a hyperspectral video object segmentation dataset, comprising hyperspectral videos and their corresponding three-channel false-color RGB videos derived from the same HSI data, with pixel-level annotations provided for each frame. The dataset contains three spectral subsets: visible (VIS), near-infrared (NIR), and red-edge near-infrared (RedNIR). Using this dataset, we developed SSAM-SAM2, a hyperspectral video object segmentation network built on Segment Anything Model 2 (SAM 2). The network includes a spectral–spatial interaction enhancement (SSIE) module designed to identify informative spectral bands, suppress interference from redundant and noisy spectral channels, and integrate hyperspectral cues with the spatial-structure features of the corresponding false-color RGB images. In addition, the network introduces an Adaptive Correlation Memory Selection (ACMS) module that regulates memory sampling and selection according to variations in target state and inter-frame feature correlation. This design reduces the influence of redundant or unreliable historical information and improves temporal memory reliability under target deformation, motion variations, and background interference. Extensive experiments show that SSAM-SAM2 outperforms the selected comparison methods on the VIS, NIR, and RedNIR subsets under the evaluated settings. These results demonstrate the value of the proposed dataset and indicate that spectral–spatial enhancement and temporal adaptive memory mechanisms contribute to improving hyperspectral video object segmentation performance.

Multispectral images contain rich spatial details and spectral information, but their large data volume places heavy demands on transmission bandwidth and storage resources. Existing multispectral image compression methods still face challenges in jointly modeling spatial and inter-spectral redundancies. In particular, similar feature-processing strategies adopted in some encoder–decoder designs, limited spectral–spatial interaction, and inadequate multi-scale modeling limit further improvements in compression efficiency. To address these issues, this paper proposes a spectral–spatial decoupling and fusion network with multi-scale perception (SSDFN) for multispectral image compression. SSDFN is built on an end-to-end learned compression framework and introduces differentiated spectral- and spatial-oriented processing pathways with tailored fusion mechanisms for the encoder and decoder. The encoder focuses on redundancy suppression and compact latent representation learning, whereas the decoder emphasizes feature interaction, information compensation, and high-fidelity reconstruction. In addition, a Multi-Scale Feature Fusion Block is developed to model complex spatial structures in multispectral remote sensing scenes. Hyperprior and context entropy models are further incorporated to improve entropy estimation. Experiments on the Landsat-8 and Sentinel-2 datasets show that SSDFN achieves competitive rate–distortion performance and provides consistent improvements in spatial reconstruction quality, perceptual quality, and spectral fidelity.

Multispectral filter array (MSFA) imaging enables low-cost, real-time, and compact spectral sensing, making it highly suitable for IoT applications such as smartphone imaging, smart agriculture, UAV-based remote sensing, and wearable diagnostics. However, due to the strong coupling among spectral response functions (SRFs), MSFA pattern, and reconstruction algorithms, achieving high-fidelity reconstruction remains a significant challenge. Conventional independent optimization of each module often leads to suboptimal results, highlighting the necessity of a co-optimized design method. To this end, we propose an end-to-end spectral reconstruction framework that jointly optimizes the SRF, MSFA pattern, and reconstruction network. In our method, the SRFs are parameterized as Gaussian functions defined by the center wavelength and full width at half maximum (FWHM). The MSFA pattern is designed using the optimal sphere-packing (OSP) method to enhance sampling uniformity and reduce artifacts. In addition, a dual-stage reconstruction pipeline is introduced: the first stage employs a Res2FFT-based U-Net with multi-scale gated attention (MGA) for spatial demosaicing, and the second stage uses an enhanced deep residual network (EDSR) for spectral super-resolution. All components are jointly trained under a hybrid loss with supervision from both multispectral and hyperspectral reconstruction. Experiments on three benchmark datasets demonstrate that our method achieves superior spatial and spectral fidelity compared to state-of-the-art methods. Furthermore, the model has been successfully deployed on an Nvidia Jetson-based IoT device, achieving real-time inference at 45 ms per frame, validating its potential for real-time spectral imaging in IoT systems.

Surgical full scene segmentation is essential for laparoscopic assistance but remains challenging due to the high visual similarity among anatomical structures, and illumination variations caused by single moving light source. Moreover, accurately segmenting thin, elongated instruments is still difficult, especially when they appear at oblique orientations. Although Mamba-based segmentation methods effectively model long-range spatial relationships through state-space updates, their 2D selective scan strategy is limited in capturing the oblique spatial distributions of surgical instruments. To address these challenges, a holistic vision Mamba block (HVMamba) is proposed with a Holistic Directional Selective Scan module (HSSD) module to integrate anisotropic spatial features from multiple directions while simultaneously modeling cross-channel dependencies to address ambiguous visual features and complex illumination. Specifically, HSSD comprises an Attention-Guided Holistic Directional Selective Scan (AHSD) for efficient integration of horizontal and oblique spatial features under the guidance of cross-coordinate relationships, and a Channel-aware Directional Selective Scan (CASD) to model bidirectional cross-channel dependencies and enhance responses in ambiguous regions. Based on HVMamba, a hierarchical hybrid network, SurgMamba, is further developed by combining an HVMamba branch with a convolutional neural network branch to jointly capture global and local representations. The two types of representations are adaptively fused by Local-Global Feature Coupling Units (LG-FCUs) in the encoder and an Attention-Aware Gating Mechanism (AGM) in the prediction head. Experiments on two public datasets demonstrate that SurgMamba achieves superior performance over state-of-the-art methods, particularly in challenging cases involving obliquely oriented thin instruments, strong specular highlights, and low-contrast tissue boundaries. Code is available at https://github.com/hailinhhh/SurgMamba.

A multispectral filter array (MSFA) is a typical snapshot spectral imaging technique, but a single sensor is limited by its narrow overall spectral range and limited number of bands. To obtain a wider spectral range and more imaging bands, the use of multiple MSFA sensors has become an effective solution. However, the existing demosaicing algorithms were designed mainly for single-MSFA systems and lack effective methods for jointly reconstructing MSFA images that are synchronously acquired from multiple sources. To address this issue, we propose a novel multi-MSFA demosaicing network, the KAN-enhanced siamese transformer pyramid network (STP-KAN). This network is built on a siamese feature pyramid architecture and uses a Kolmogorov–Arnold Network (KAN)-based half-shuffle multi-head self-attention as its basic module, effectively leveraging the long-range modeling capabilities of transformers and the powerful nonlinear representation ability of the KAN. A cross-attention mechanism is utilized to efficiently share features among multi-MSFA images. A spatial-spectral attention mechanism is embedded into the skip connections of the network to fuse deep spatial and spectral features. MSFA demosaicing experiments conducted on the ARAD-1K open-source natural scene dataset and our self-built AVIRIS-NG remote sensing dataset demonstrate the superior performance of our proposed method. The source code is available at https://github.com/Stephen-future/STP-KAN.

Satellite-based aircraft monitoring is an important complement to ground surveillance systems, providing strong support for the safe, efficient, and reliable operation of global aviation. Most existing aircraft detection datasets are derived from still satellite imagery, making it difficult to detect flying aircraft. Although video satellite imagery can provide motion cues, its spatial coverage is limited, making it challenging to capture flying aircraft targets that are sparsely distributed over wide areas. Each Sentinel-2 satellite image covers a width of hundreds of kilometers, providing favorable conditions for monitoring flying aircraft. Beyond this, the physical design of its multispectral instruments (MSIs) induces parallax effects for moving objects in multispectral imagery, enabling a novel approach for the detection of flying aircraft. We construct a flying aircraft detection dataset (S2Aircraft) based on Sentinel-2 satellite multispectral imagery with a spatial resolution of 10 m. The dataset is annotated with oriented bounding boxes (OBBs) and includes both RGB and NIR spectral bands. Additionally, we design an efficient flying aircraft detection network (FADet), which maps input images to a high-dimensional nonlinear feature space while maintaining low computational complexity. Moreover, for single-class object detection tasks, the model employs a semi-decoupled head to achieve efficient detection. Finally, a loss function is specifically designed according to the geometric characteristics of targets in the S2Aircraft dataset, significantly improving the accuracy and stability of oriented object detection. Extensive experiments demonstrate the effectiveness and advancement of our FADet. Specifically, on our S2Aircraft dataset, FADet achieves competitive performance reaching 2.6 GFLOPs and 96.3% mAP$_{50}$ on the S2Aircraft dataset. On two public datasets, HRSC2016 and CORS-ADD, FADet achieves mAP$_{50}$ of 90.90% and 94.16%, respectively. The dataset and codes are available at https://github.com/maonanmaonan/FADet.

Compared to conventional RGB images, hyperspectral images offer a more comprehensive range of spectral information, encompassing both visible and infrared bands. This enhanced spectral information facilitates trackers in effectively differentiating the target object from background clutter, realizing more robust recognition. However, hyperspectral video cameras exhibit variability in terms of spectral ranges and band numbers, producing distinct modalities. Developing specialized networks for each modality proves to be inefficient and time-consuming, while networks trained for one modality struggle to perform well on others, generating unsatisfactory outcomes. To confront these challenges encountered in various tracking scenarios, HyperTrack is proposed as a unified object tracking network tailored for hyperspectral videos. The proposed HyperTrack can be employed individually for single object tracking across three different modalities of hyperspectral videos: near infrared (NIR), visible (VIS), and red-to-near infrared (RedNIR). Specifically, since hyperspectral data have multiple bands, a band gate module is introduced into the network to enable it to select bands from hyperspectral images, thereby reducing the dimensionality for hyperspectral images. Furthermore, in order to effectively utilize the variability amongst the three different modal data, a computationally simple band embedding module is introduced to improve the object tracking performance of different modal hyperspectral videos. Additionally, a hybrid attention module is devised to efficiently extract and interact features between the template and search at each stage. As a unified network, HyperTrack achieves state-of-the-art comprehensive results across three different types of hyperspectral videos, particularly excelling with VIS and NIR type data. The code and models are publicly available at https://github.com/supertyd/HyperTrack.

Hyperspectral videos contain richer spectral and physical features than RGB videos and thus have greater potential for use in object tracking. The mainstream hyperspectral object tracking approach involves the integration of multiple RGB-based video tracking models. Although ensembles of multiple models can effectively utilize spectral information and improve tracker performance, this approach has high computational complexity, making it difficult to meet the real-time requirements of video object tracking. To bridge the gap, we propose a new hyperspectral object tracking framework (HotMoE) based on Mixture-of-Experts (MoE). HotMoE leverages a divide-and-conquer strategy, where only a subset of expert models is computed for each input, reducing computational complexity while maintaining performance. In this paper, we first design a splitter to group multiple spectral bands into multiple false-color images based on spectral correlations. Then, we design a hyperspectral MoE router that can adaptively learn to aggregate spectral image feature information and route it to suitable experts. Different experts can handle various scenarios, and HotMoE effectively utilizes the capabilities of different experts to obtain better overall performance. Compared with previous state-of-the-art hyperspectral object tracking networks, our model has significantly reduced inference time and performs well, with a processing speed of 43.7 FPS and an AUC of 0.704 with the HOT2022 dataset. The codes and models are available at https://github.com/supertyd/hotmoe.

The real-time performance and precision requirements posed by the ranging method for proximity fuze determines the effect and hit accuracy of the explosive device on which it is mounted. This further demands not only high spectrum resolution near the burst point for the fuze, but also the ability to monitor the spectrum of the entire observation area. This paper proposes a ranging method for proximity fuze based on warped fast Fourier transform (WFFT). By selecting appropriate all-pass warping factors, our proposed method can improve the spectrum precision within a specific spectrum region observed without increasing signal processing points. By fitting non-uniform filter banks, the diffused energy of target object in high-frequency region is accumulated, which can be regarded as improving the signal-to-noise ratio (SNR) in high-frequency region, high-frequency information can be accurately captured, thereby facilitating real-time ranging of proximity fuze with higher precision. The experimental results have verified the feasibility and effectiveness of our proposed method in practical proximity fuze ranging.