InBRwSANet: Self-attention based parallel inverted residual bottleneck architecture for human action recognition in smart cities
Open Access
- 27 May 2025
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 20 (5) , e0322555
Abstract
Human Action Recognition (HAR) has grown significantly because of its many uses, including real-time surveillance and human-computer interaction. Various variations in routine human actions make the recognition process of action more difficult. In this paper, we proposed a novel deep learning architecture known as Inverted Bottleneck Residual with Self-Attention (InBRwSA). The proposed architecture is based on two different modules. In the first module, 6-parallel inverted bottleneck residual blocks are designed, and each block is connected with a skip connection. These blocks aim to learn complex human actions in many convolutional layers. After that, the second module is designed based on the self-attention mechanism. The learned weights of the first module are passed to self-attention, extract the most essential features, and can easily discriminate complex human actions. The proposed architecture is trained on the selected datasets, whereas the hyperparameters are chosen using the particle swarm optimization (PSO) algorithm. The trained model is employed in the testing phase for the feature extraction from the self-attention layer and passed to the shallow wide neural network classifier for the final classification. The HMDB51 and UCF 101 are frequently used as action recognition standard datasets. These datasets are chosen to allow for meaningful comparison with earlier research. UCF101 dataset has a wide range of activity classes, and HMDB51 has varied real-world behaviors. These features test the generalizability and flexibility of the presented model. Moreover, these datasets define the evaluation scope within a particular domain and guarantee relevance to real-world circumstances. The proposed technique is tested on both datasets, and accuracies of 78.80% and 91.80% were achieved, respectively. The ablation study demonstrated that a margin of error value of 70.1338 ± 3.053 (±4.35%) and 82.7813 ± 2.852 (±3.45%) for the confidence level 95%,1.960σx̄ is obtained for HMDB51 and UCF datasets respectively. The training time for the highest accuracy for HDMB51 and UCF101 is 134.09 and 252.10 seconds, respectively. The proposed architecture is compared with several pre-trained deep models and state-of-the-art (SOTA) existing techniques. Based on the results, the proposed architecture outperformed existing techniques.Keywords
Funding Information
- Deanship of Scientific Research, King Khalid University (RGP.2/379/45)
- National Research Foundation of Korea (RS-2023-00218176)
This publication has 42 references indexed in Scilit:
- Shots segmentation-based optimized dual-stream framework for robust human activity recognition in surveillance videoAlexandria Engineering Journal, 2024
- A survey on intelligent human action recognition techniquesMultimedia Tools and Applications, 2023
- Image expression of time series data of wearable IMU sensor and fusion classification of gymnastics actionExpert Systems with Applications, 2023
- A dynamic fusion of features from deep learning and the HOG-TOP algorithm for facial expression recognitionMultimedia Tools and Applications, 2023
- Human Activity Recognition (HAR) Using Deep Learning: Review, Methodologies, Progress and Future Research DirectionsArchives of Computational Methods in Engineering, 2023
- TransCNN: Hybrid CNN and transformer mechanism for surveillance anomaly detectionEngineering Applications of Artificial Intelligence, 2023
- Transfer learning and fine‐tuned transfer learning methods' effectiveness analyse in the CNN‐based deep learning modelsConcurrency and Computation: Practice and Experience, 2022
- TCLR: Temporal contrastive learning for video representationComputer Vision and Image Understanding, 2022
- Deep learning network model based on fusion of spatiotemporal features for action recognitionMultimedia Tools and Applications, 2022
- Attention-Based Temporal Weighted Convolutional Neural Network for Action Recognition2018