Hierarchical Cross-Modal Fusion Model for Field Vehicle Classification
- 15 May 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 75 (00189456) , 9520515
Abstract
Field Vehicle classification is an important task for unattended ground sensor systems. This paper proposes a multimodal fusion-based classification method. Based on field measurement data collected in field environments, each signal sample contains continuous acoustic and seismic signals together with discrete image signals, which leads to information imbalance between the image modality and the acoustic-seismic modalities. To address this issue in multimodal fusion, a Hierarchical Cross-Modal Fusion (HCMF) method is proposed. First, shallow and deep convolutional features are extracted for each modality using a convolutional network. Subsequently, a two-level fusion strategy is performed using the proposed HCMF framework. In the first-level fusion, three intra-modal transformer models are employed to fuse the two convolutional features within each modality, and the outputs of the three modalities are combined into a hybrid modality feature through confidence-weighted fusion. In the second-level fusion, a inter-modal transformer model processes the hybrid modality feature to enable cross-modal information interaction, and the resulting representation is used for classification. Based on a small-scale dataset constructed from field measurement data, a series of experiments are conducted, including model generalization evaluation, ablation studies, modality absence analysis, and algorithm comparisons. Experimental results demonstrate that the proposed model exhibits generalization capability, achieving a macro F1-score of 98.45% with 1.77 million parameters and 81.19 million FLOPs. The proposed method effectively balances classification performance and computational complexity, providing a potential solution for embedded applications in field vehicle classification.Keywords
This publication has 37 references indexed in Scilit:
- A Style-Pooling Vision Transformer for Underwater Object Classification Based on Audio-Visual Image Fusion2024
- Research on multimodal information fusion dynamic target recognition method for autonomous driving2023
- AI Choreographer: Music Conditioned 3D Dance Generation with AIST++2021
- An EMD-Based Micro-Doppler Signature for Moving Vehicles Classification2019
- Single-Mode Wild Area Surveillance Sensor With Ultra-Low Power Design Based on Microphone ArrayIEEE Access, 2019
- Vehicle Type Classification Based on Acoustic Signals Using Denoised MFCC2018
- Research on armored vehicle classification based on MFCC and SVM2017
- Vehicle detection and classification from acoustic signal using ANN and KNN2013
- Computational techniques for classification of military vehicles using seismic signatures2012
- Vehicle identification based on self-organizing artificial neural network2002