Deep Fusion Module for Video Action Recognition
- 3 April 2024
- journal article
- research article
- Published by World Scientific Pub Co Pte Ltd in Journal of Circuits, Systems and Computers
- Vol. 33 (14)
Abstract
In video action recognition, effective spatiotemporal modeling is crucial. However, traditional two-stream methods face challenges in integrating spatial information from RGB images and temporary information from optical flow without long-range temporal modelling. To address these limitations, we propose the Deep Fusion Module (DFM), which focuses on the deep fusion of spatial and temporal information and consists of two components. First, we propose an Attention Fusion Module (AFM) to effectively fuse the shallow features obtained from a two-stream network, thereby facilitating the integration of spatial and temporal information. Next, we incorporate a SpatioTemporal Module (STM), comprising a ConvGRU and a 1×1 convolution, to model long-range temporal dependency and fuse spatial-temporal features. Experiments on the UCF101 dataset show that our method achieves 96.5% accuracy, outperforming baseline two-stream models by 0.3%.Keywords
Funding Information
- NSFC (62176027)
This publication has 28 references indexed in Scilit:
- SVFormer: Semi-supervised Video Transformer for Action Recognition2023
- DirecFormer: A Directed Attention in Transformer Approach to Robust Action Recognition2022
- Video Swin Transformer2022
- Temporal Relational Reasoning in Videos2018
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition2016
- Large-Scale Video Classification with Convolutional Neural Networks2014
- HMDB: A large video database for human motion recognition2011
- Action recognition by dense trajectories2011
- ImageNet: A large-scale hierarchical image database2009