Retain, Blend, and Exchange: A Quality-Aware Spatial-Stereo Fusion Approach for Event Stream Recognition
- 12 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 8926-8939
Abstract
Current event stream-based pattern recognition models typically present the event stream as the point cloud, voxel, image, and the like, and formulate multiple deep neural networks to acquire their features. Although considerable results can be achieved in simple cases, however, the performance of the model might be restricted by monotonous modality expressions, sub-optimal fusion, and readout mechanisms. In this article, we put forward a novel dual-stream framework for event stream-based pattern recognition through differentiated fusion, which is called EFV++. It models two common event representations simultaneously, i.e., event images and event voxels. The spatial and three-dimensional stereo information can be separately learned by making use of Transformer and Graph Neural Network (GNN). We believe the features of each representation still contain both efficient and redundant features and a sub-optimal solution may be obtained if we directly fuse them without differentiation. Thus, we divide each feature into three levels and retain high-quality features, blend medium-quality features, and exchange low-quality features. The enhanced dual features will be provided to the fusion Transformer together with bottleneck features. In addition, we introduce a novel hybrid interaction readout mechanism to enhance the diversity of features as final representations. Comprehensive experiments validate that the framework we have proposed attains cutting-edge performance on a variety of extensively utilized event stream-based classification datasets. Particularly, we have realized a freshly pioneering performance on the Bullying10 k dataset, precisely 90.51%, and this outpaces the runner-up by $+2.21\%$.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62102205, 62027804, 62332002, 62027804, 61825101, 62302041, U24A20342)
- Anhui Provincial Natural Science Foundation - Outstanding Youth (2408085Y032)
- Postdoctoral Fellowship Program of CPSF (GZC20240840)
- Major Key Project of PCL (PCL2021A13)
- Project of Peng Cheng Laboratory (PCL2023A08)
This publication has 65 references indexed in Scilit:
- Revisiting color-event based tracking: A unified network, dataset, and metricPattern Recognition, 2025
- EventBind: Learning a Unified Representation to Bind Them All for Event-Based Open-World Understanding2024
- Semantic-aware frame-event fusion based pattern recognition via large vision–language modelsPattern Recognition, 2024
- Learning Bottleneck Transformer for Event Image-Voxel Feature Fusion Based Classification2023
- Large-scale Multi-modal Pre-trained Models: A Comprehensive SurveyMachine Intelligence Research, 2023
- Pedestrian attribute recognition: A surveyPattern Recognition, 2022
- A Differentiable Recurrent Surface for Asynchronous Event-Based Data2020
- URIE: Universal Image Enhancement for Visual Recognition in the Wild2020
- Deep Residual Learning for Image Recognition2016
- ImageNet: A large-scale hierarchical image database2009