Abstract
Field Vehicle classification is an important task for unattended ground sensor systems. This paper proposes a multimodal fusion-based classification method. Based on field measurement data collected in field environments, each signal sample contains continuous acoustic and seismic signals together with discrete image signals, which leads to information imbalance between the image modality and the acoustic-seismic modalities. To address this issue in multimodal fusion, a Hierarchical Cross-Modal Fusion (HCMF) method is proposed. First, shallow and deep convolutional features are extracted for each modality using a convolutional network. Subsequently, a two-level fusion strategy is performed using the proposed HCMF framework. In the first-level fusion, three intra-modal transformer models are employed to fuse the two convolutional features within each modality, and the outputs of the three modalities are combined into a hybrid modality feature through confidence-weighted fusion. In the second-level fusion, a inter-modal transformer model processes the hybrid modality feature to enable cross-modal information interaction, and the resulting representation is used for classification. Based on a small-scale dataset constructed from field measurement data, a series of experiments are conducted, including model generalization evaluation, ablation studies, modality absence analysis, and algorithm comparisons. Experimental results demonstrate that the proposed model exhibits generalization capability, achieving a macro F1-score of 98.45% with 1.77 million parameters and 81.19 million FLOPs. The proposed method effectively balances classification performance and computational complexity, providing a potential solution for embedded applications in field vehicle classification.