Action Recognition in Dark Videos Using Spatio-Temporal Features and Bidirectional Encoder Representations From Transformers
- 17 November 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. 4 (6) , 1461-1471
Abstract
Several research works have been developed in the area of action recognition. Unfortunately, when these algorithms are applied to low-light or dark videos, their performances are highly affected and found to be very poor or fall rapidly. To address the issue of improving the performance of action recognition in dark or low-light videos; in this article, we have developed an efficient deep 3D CNN based action recognition model. The proposed algorithm follows two-stages for action recognition. In the first stage, the low-light videos are enhanced using Zero-Reference Deep Curve Estimation (Zero-DCE), followed by the min-max sampling algorithm. In the latter stage, we propose an action classification network to recognize the actions in the enhanced videos. In the proposed action classification network, we explored the capabilities of the $R(2+1)D$ for spatio-temporal feature extraction. The model's overall generalization performance depends on how well it can capture long-range temporal structure in videos, which is essential for action recognition. So we have used a Graph convolutional network (GCN) on the top of R(2+1)D as our video feature encoder which captures long-term temporal dependencies of the extracted features. Finally, a Bidirectional Encoder Representations from Transformers (BERT) is adhered to classify the actions from the 3D features extracted from the enhanced video scenes. The effectiveness of the proposed action recognition scheme is verified on ARID V1.0 and ARID V1.5 datasets. It is observed that the proposed algorithm is able to achieve $96.60\%$ and $99.88\%$ as Top-1 and Top-5 accuracy, respectively, on ARID V1.0 dataset. Similarly, on ARID V1.5, the proposed algorithm is able to achieve $86.93\%$ and $99.35\%$ as Top-1 and Top-5 accuracies, respectively. To corroborate our findings, we have compared the results obtained by the proposed scheme with those of fifteen state-of-the-art action recognition techniques.Keywords
Funding Information
- Ministry of Electronics and Information technology (4(16)/2019-ITEA)
This publication has 31 references indexed in Scilit:
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- Interpretable 3D Human Action Analysis with Temporal Convolutional Networks2017
- Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding2016
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition2016
- Deep Residual Learning for Image Recognition2016
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- Robust Video Surveillance for Fall Detection Based on Human Shape DeformationIEEE Transactions on Circuits and Systems for Video Technology, 2011
- Exposure Fusion: A Simple and Practical Alternative to High Dynamic Range PhotographyComputer Graphics Forum, 2009
- A Human Action Recognition System for Embedded Computer Vision Application2007
- A spatial processor model for object colour perceptionJournal of the Franklin Institute, 1980