Enhancing Human Action Recognition and Violence Detection Through Deep Learning Audiovisual Fusion
Preprint
Abstract
This paper proposes a hybrid fusion-based deep learning approach based on two different modalities, audio and video, to improve human activity recognition and violence detection in public places. To take advantage of audiovisual fusion, late fusion, intermediate fusion, and hybrid fusion-based deep learning (HFBDL) are used and compared. Since the objective is to detect and recognize human violence in public places, Real-life violence situation (RLVS) dataset is expanded and used. Simulating results of HFBDL show 96.67\% accuracy on validation data, which is more accurate than the other state-of-the-art methods on this dataset. To showcase our model's ability in real-world scenarios, another dataset of 54 sounded videos of both violent and non-violent situations was recorded. The model could successfully detect 52 out of 54 videos correctly. The proposed method shows a promising performance on real scenarios. Thus, it can be used for human action recognition and violence detection in public places for security purposes.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- Crime scene classification from skeletal trajectory analysis in surveillance settingsEngineering Applications of Artificial Intelligence, 2024
- ML: Early Breast Cancer DiagnosisCurrent Problems in Cancer: Case Reports, 2024
- Machine learning techniques for classifying dangerous asteroidsMethodsX, 2023
- Temporal-spatial convolutional residual network for decoding attempted movement related EEG signals of subjects with spinal cord injuryComputers in Biology and Medicine, 2023
- Human skeletons and change detection for efficient violence detection in surveillance videosComputer Vision and Image Understanding, 2023
- A novel tree pattern-based violence detection model using audio signalsExpert Systems with Applications, 2023
- Video surveillance using deep transfer learning and deep domain adaptation: Towards better generalizationEngineering Applications of Artificial Intelligence, 2022
- Multimodal fusion methods with deep neural networks and meta-information for aggression detection in surveillanceExpert Systems with Applications, 2022
- Multi-view learning for visual violence recognition with maximum entropy discrimination and deep featuresInformation Fusion, 2019
- Fusion strategies for recognition of violence actions2017