Jie Zhu

Cross-modal remote sensing object detection holds significant potential for around-the-clock applications. However, the modality differences between cross-modal data and the degradation of feature quality under adverse weather conditions limit detection performance. To address these challenges, this paper presents a novel cross-modal remote sensing object detection framework designed to overcome two critical challenges in around-the-clock applications: (1) significant modality disparities between visible light, infrared, and synthetic aperture radar data, and (2) severe feature degradation under adverse weather conditions including fog, and nighttime scenarios. Our primary contributions are as follows: First, we develop a multi-scale feature extraction module that employs a hierarchical convolutional architecture to capture both fine-grained details and contextual information, effectively compensating for missing or blurred features in degraded visible-light images. Second, we introduce an innovative feature interaction module that utilizes cross-attention mechanisms to establish long-range dependencies across modalities while dynamically suppressing noise interference through adaptive feature selection. Third, we propose a feature correction fusion module that performs spatial alignment of object boundaries and channel-wise optimization of global feature consistency, enabling robust fusion of complementary information from different modalities. The proposed framework is validated on visible light, infrared, and SAR modalities. Extensive experiments on three challenging datasets (LLVIP, OGSOD, and Drone Vehicle) demonstrate our framework’s superior performance, achieving state-of-the-art mean average precision scores of 66.3%, 58.6%, and 71.7%, respectively, representing significant improvements over existing methods in scenarios with modality differences or extreme weather conditions. The proposed solution not only advances the technical frontier of cross-modal object detection but also provides practical value for mission-critical applications such as 24/7 surveillance systems, military reconnaissance, and emergency response operations where reliable around-the-clock detection is essential.

To detect and recognize small-size and submerged complex background targets in infrared images, we combine a dynamic receptive field fusion strategy and a multi-scale feature fusion mechanism to improve the detection performance of small targets significantly. The space-to-depth convolution module is introduced as a downsampling layer in the backbone first and achieves the same sampling effect. More detailed information is retained at the same time. Thus, the model’s detection capability for small targets has been enhanced. Then, the pyramid level 2 feature map with minimum receptive field and maximum resolution is added to the neck, which reduces the loss of positional information during feature sampling. Furthermore, x-small detection heads are added, the understanding of the overall characteristics and structure of the target is enhanced much more, and the representation and localization of small targets have been improved. Finally, the cross-entropy loss function in the original network model is replaced by an adaptive threshold focal loss function, forcing the model to allocate more attention to target features. The above methods are based on a public tool, the eighth version of You Only Look Once (YOLO) improved, it is named SPT–YOLO (SPDConv + P2 + Adaptive Threshold + YOLOV8s) in this paper. Some experiments on datasets such as infrared small object detection (IR-SOD) and infrared small target detection 1K(IRSTD-1K), etc. have been executed to verify the proposed algorithm; and the mean average precision of 94.0% and 69% under the condition of threshold at 0.5 and over a range from 0.5 to 0.95 is obtained, respectively. The results show that the proposed method achieves the best performance of infrared small target detection compared to existing methods.

In order to provide better geographic information services on time, a detection technique on the variation of remote sensing image has been investigated in this paper. Based on multiscale feature multiplex, a consistent regularization method together with a semi-supervised strategy has been proposed, which can solve the problems of unclear change edges and the issue of missing detection of very small targets. By constructing a learning training model of detection network, the performances have been enhanced with limited labeled data and effectively handle change detection tasks in complex scenes. The network consists of two parts. One part is the supervised multiscale feature fusion and multiplex, which enhances feature expression capabilities using channel and spatial attention mechanisms. We extract variation features through complex multiscale feature fusion and reuse strategies to alleviate unclear change edges and the omission of very small targets. The other part is the consistency regularization method for semi-supervised learning model, which adopts an alternating training among models. The prediction differences have been minimized, then updated model parameters. This constructed model of detection network has been verified at different labeling ratios on the datasets such as Wuhan university, China, building change detection and google earth imagery Guangzhou, China, change detection, etc. The experimental results show that this proposed technique demonstrates the best overall performance in complex situations with small dataset labeling ratios among existing detection methods.

To detect and recognize small size and submerged complex background target in infrared images, we combine a dynamic receptive field fusion strategy and a multi-scale feature fusion mechanism to improve the detection performance of small targets significantly. The space-to-depth convolution module is introduced as a downsampling layer in the backbone firstly, and achieved the same sampling effect. More detailed information retained at the same time. Thus the model's detection capability for small targets has been enhanced. Then the pyramid level 2 feature map with minimum receptive field and maximum resolution is added to the neck, which reduces the loss of positional information during feature sampled. Furthermore, x-small detection head are added, The understanding of the overall characteristics and structure of the target is enhanced much more, and the representation and localization of small targets has been improved. Finally, the cross-entropy loss function in the original network model is replaced by an adaptive threshold focal loss function, forcing the model to allocate more attention to target features. Above methods base on a public tool, the 8th version of You Only Look Once (YOLO) improved, it is named SPT-YOLO in this paper. Some experiments on datasets such as infrared small object detection (IR-SOT) and infrared small target detection 1K (IRSTD-1K), etc, have been executed to verify the proposed algorithm; and the mean average precision of 94.0% and 69% under condition of threshold at 0.5 and over a range from 0.5 to 0.95 is obtained, respectively. The results show that the proposed method achieves the best performance of infrared small target detection compared to existing methods.

Remote sensing image change detection is a key application technology in remote sensing. In recent years, the development of deep learning has brought new opportunities for remote sensing image change detection. However, to achieve high change detection accuracy, large-scale, good-quality labeled data is required to train the model, which is often difficult to obtain and costly. This paper proposes a semi-supervised remote sensing image change detection network based on consistent regularization for multi-scale feature fusion and reuse (SCMFM-CDNet), which can enhance change detection performance with limited labeled data. Specifically, the SCMFM-CDNet consists of two parts, the supervised stage multi-scale feature fusion and reuse network (MFM-CDNet) and the consistency regularization method of semi-supervised learning. The MFM-CDNet network uses channel attention mechanism and spatial attention mechanism to enhance the expression ability of features, and gradually completes the extraction and detection of change features through multi-scale feature fusion and reuse strategy. The consistency regularization method of semi-supervised learning adopts the teacher model and student model for alternate training, updating model parameters by minimizing the differences in their predictions, thus achieving semi-supervised learning. This paper conducted extensive experiments on three public datasets, LEVIR-CD, WHU-CD, and GoogleGZ-CD. Under the three labeling data ratios of 5%, 10%, and 20%, the F1 scores of the SCMFM-CDNet network on the LEVIR-CD dataset were 88.62, 90.78, and 90.63, respectively. On the WHU-CD dataset, the F1 scores were 88.40, 90.17, and 90.83, respectively. On the GoogleGZ-CD dataset, the F1 scores were 81.44, 82.36, and 85.40, respectively. The results indicate that the proposed network outperforms existing semi-supervised change detection methods in terms of comprehensive performance. In addition, this paper also conducted ablation experiments to evaluate the contribution of the global context module (GCM) and feature fusion reuse module (FFM) in the MFM-CDNet network. The experimental results show that these two modules are crucial for improving network performance.

Flexible piezoresistive sensors based on the mechanism of interfacial contact resistance change are receiving increasing attention in the fields of human-computer interaction, health monitoring, and behavior tracking. However, the high cost and complex manufacturing process limit the wide application and development of these flexible piezoresistive sensors. Here, a novel carbon nanotubes@fabric (CNTs@fabric)/Ni@polyurethane (Ni@PU) piezoresistive sensor (CNPS) with low-cost, simple-preparation and high-sensitivity was proposed. The effective contact area is obtained by synergizing the woven micro-convex structure of the fabric, the large specific surface area of the CNTs and the porous three dimensional electrodes. Within the small pressure (0–9.52 kPa) effect, the area of connection with the electrodes to the active layer plays a dominant role, resulting in a sensitivity of up to 6.39 kPa⁻¹ for CNPS. In the high pressure region (9.52–44.92 kPa), where internal mechanism of change in the sensitive material dominants, the CNPS has a response time of 85 ms at a constant pressure of 28.31 kPa. Considering the excellent output electrical performances, a variety of body movements could be detected by fixing the CNPS to different joints. Significantly, the designed intelligent object recognition system implemented by the combination of matrix stress detection module and residual neural network (ResNet) algorithm has a recognition accuracy of 99.26%. Enhancing the interfacial contact resistance change mechanism using a simple fabrication process offers a promising strategy for the rapid development of flexible piezoresistive sensors.

The array of tactile information processing capabilities is an important index for modern intelligent devices advancing toward a humanoid form, and it greatly improves the recognition of different objects in human‐computer interactions. Herein, a deep‐learning‐assisted intelligent grasping recognition system based on a piezoresistive sensing glove, hardware conditioning, and acquisition circuits, and a multibranch deep‐capsule network is reported. Owing to the multiscale 3D structure of carbon nanotube (CNTs)/carbon fiber (CFs) embedded in polydimethylsiloxane (PDMS), the piezoresistive sensing glove is highly sensitive to the pressure exerted by external objects. The acquired signals are reflected on a hand‐like background map, and a combination of multiple subgraphs is used to build the dataset. A multibranch deep‐capsule network is constructed to encode spatial information while realizing object recognition with an accuracy of 99.4%. Therefore, the proposed intelligent grasping recognition system possesses good human‐robot interaction capabilities, providing a new approach for the development of intelligent robots in the field of perceptual recognition applications.

In response to the growing demand for high-sensitivity accelerometers in vector hydrophones, a piezoelectric MEMS accelerometer (PMA) was proposed, which has a four-cantilever beam integrated inertial mass unit structure, with the advantages of being lightweight and highly sensitive. A theoretical energy harvesting model was established for the piezoelectric cantilever beam, and the geometric dimensions and structure of the microdevice were optimized to meet the vibration pickup conditions. The sol-gel and annealing technology was employed to prepare high-quality PZT thin films on silicon substrate, and accelerometer microdevices were manufactured by using MEMS technology. Furthermore, the MEMS accelerometer was packaged for testing on a vibration measuring platform. Test results show that the PMA has a resonant frequency of 2300 Hz. In addition, there is a good linear relationship between the input acceleration and the output voltage, with V = 8.412a − 0.212. The PMA not only has high sensitivity, but also has outstanding anti-interference ability. The accelerometer structure was integrated into a vector hydrophone for testing in a calibration system. The results show that the piezoelectric vector hydrophone (PVH) has a sensitivity of –178.99 dB@1000 Hz (0 dB = 1 V/μPa) and a bandwidth of 20~1100 Hz. Meanwhile, it exhibits a good “8” shape directivity and consistency of each channel. These results demonstrate that the piezoelectric MEMS accelerometer has excellent capabilities suitable for use in vector hydrophones.