Yan Wang

Acoustic and image sensors in unattended ground sensor systems often operate with different sampling rates, trigger mechanisms, and fields of view, so their observations are asynchronous and lack instance-level correspondence. In addition, web images used to supplement scarce field annotations exhibit a substantial domain shift from target-domain images. This paper proposes a three-stream joint learning framework that integrates reliability-weighted image-domain adaptation, reliability-aware dynamic bidirectional prototype distillation, and balanced complementary modality dropout. A lightweight shared backbone processes labeled acoustic spectra, labeled mixed-source images, and unlabeled target-domain images. Target pseudo-label reliability is estimated by a main classifier and two auxiliary classifiers and is used to weight marginal and class-conditional distribution alignment. Acoustic and image class prototypes transfer category-level knowledge without constructing artificial instance pairs, while direction-specific distillation weights are dynamically assigned according to classification reliability, prototype availability, and prototype stability. Experiments on 148,038 acoustic spectra and 167,754 images yield acoustic, image, and overall accuracies of 92.02\%, 95.26\%, and 93.79\%, respectively, exceeding the best corresponding baseline results by 1.49, 1.24, and 2.47 percentage points. The deployment model contains 123.40 K parameters and requires 0.47 MiB in FP32, supporting resource-constrained field deployment.

This paper presents an end-to-end multimodal fusion approach called powerset fusion network (PFN), based on a convolutional neural network (CNN), which is used for acoustic-seismic moving target classification in unattended ground sensor (UGS) systems. We first introduce the powerset fusion to fuse different modalities sufficiently. Then fusing representations at all abstraction levels is explored by using the proposed fully hybrid framework. We also examine how to use multi-task learning (MTL) to obtain better representations, overcome missing modalities and save embedded storage space. Finally, we present a novel multimodal cost function to accelerate the learning procedure. The experimental results based on the field dataset show that the performance of PFN is superior to that of the popular deep learning fusion systems.

Seismic vibration signatures are strong criteria to recognize moving ground targets in unattended ground sensor (UGS) systems. However, it is a challenging task because of the complexity of seismic waves and their high dependency on the underlying geology. In order to approach this problem, this paper proposes a novel method called “VibCNN” based on convolutional neural networks (CNNs). Instead of preprocessing signals to extract features, the proposed model takes raw waveforms as input. Another characteristic of the model is that it can handle very short input, which only contains 1024 sample points. The experimental results show that the model yields performance much better than benchmarks and generalizes quite well across different geological types. To further improve the performance of VibCNN, we introduce two auxiliary input channels based on seismic signals and add each auxiliary channel to the input layer of VibCNN separately. Furthermore, we explore different fusion rules of the auxiliary channels at three levels: sample level, feature level, and decision level. The best result achieves relative improvement of 2.05%. In addition, data augmentation for seismic data has not been deeply investigated yet. Thus, we conduce a data augmentation experiment to explore the influence of different augmentation techniques on the performance of the model. The appropriate augmentation improves the accuracy of the model from 93.44% to 95.20%.