Human Action Representation Learning Using an Attention-Driven Residual 3DCNN Network
Open Access
- 31 July 2023
- journal article
- research article
- Published by MDPI AG in Algorithms
- Vol. 16 (8) , 369
Abstract
The recognition of human activities using vision-based techniques has become a crucial research field in video analytics. Over the last decade, there have been numerous advancements in deep learning algorithms aimed at accurately detecting complex human actions in video streams. While these algorithms have demonstrated impressive performance in activity recognition, they often exhibit a bias towards either model performance or computational efficiency. This biased trade-off between robustness and efficiency poses challenges when addressing complex human activity recognition problems. To address this issue, this paper presents a computationally efficient yet robust approach, exploiting saliency-aware spatial and temporal features for human action recognition in videos. To achieve effective representation of human actions, we propose an efficient approach called the dual-attentional Residual 3D Convolutional Neural Network (DA-R3DCNN). Our proposed method utilizes a unified channel-spatial attention mechanism, allowing it to efficiently extract significant human-centric features from video frames. By combining dual channel-spatial attention layers with residual 3D convolution layers, the network becomes more discerning in capturing spatial receptive fields containing objects within the feature maps. To assess the effectiveness and robustness of our proposed method, we have conducted extensive experiments on four well-established benchmark datasets for human action recognition. The quantitative results obtained validate the efficiency of our method, showcasing significant improvements in accuracy of up to 11% as compared to state-of-the-art human action recognition methods. Additionally, our evaluation of inference time reveals that the proposed method achieves up to a 74× improvement in frames per second (FPS) compared to existing approaches, thus showing the suitability and effectiveness of the proposed DA-R3DCNN for real-time human activity recognition.Keywords
Funding Information
- Air Force Office of Scientific Research (FA9550-22-1-0040)
This publication has 68 references indexed in Scilit:
- Beyond Frame-level CNN: Saliency-Aware 3-D CNN With LSTM for Video Action RecognitionIEEE Signal Processing Letters, 2016
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition2016
- Deep Residual Learning for Image Recognition2016
- Hierarchical Clustering Multi-Task Learning for Joint Human Action Grouping and RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- P-CNN: Pose-Based CNN Features for Action Recognition2015
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- Long-term recurrent convolutional networks for visual recognition and description2015
- Large-Scale Video Classification with Convolutional Neural Networks2014
- A lightweight dynamic optimization methodology and application metrics estimation model for wireless sensor networksSustainable Computing: Informatics and Systems, 2013
- Recognizing 50 human action categories of web videosMachine Vision and Applications, 2012