Joint Multi-Scale Residual and Motion Feature Learning for Action Recognition
Open Access
- 23 September 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 701-706
Abstract
For action recognition, two-stream networks consisting of RGB and optical flow has been widely used, showing high recognition accuracy. However, optical flow computation is time-consuming and requires a large amount of storage space, and the recognition efficiency is very low. To alleviate this problem, we propose an Adaptive Multi-Scale Residual (AMSR) module and a Long Short Term Motion Squeeze (LSMS) module, which are inserted into the 2D convolutional neural network to improve the accuracy of action recognition and achieve a balance of accuracy and speed. The AMSR module adaptively fuses multi-scale feature maps to fully utilize the semantic information provided by deep feature maps and the detailed information provided by shallow feature maps. The LSMS module is a learnable lightweight motion feature extractor for learning long-term motion features of adjacent and non-adjacent frames, thus replacing the traditional optical flow and improving the accuracy of action recognition. Experimental results on UCF-101 and HMDB-51 datasets demonstrate that the method proposed in this paper achieves competitive performance compared to state-of-the-art methods with only a small increase in parameters and computational cost.Keywords
This publication has 21 references indexed in Scilit:
- TMF: Temporal Motion and Fusion for action recognitionComputer Vision and Image Understanding, 2021
- 3D convolutional networks with multi-layer-pooling selection fusion for video classificationMultimedia Tools and Applications, 2021
- MotionSqueeze: Neural Motion Feature Learning for Video Understanding2020
- FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks2017
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition2016
- Deep Residual Learning for Image Recognition2016
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- HMDB: A large video database for human motion recognition2011
- Determining optical flowArtificial Intelligence, 1981