Transformer and Adaptive Threshold Sliding Window for Improving Violence Detection in Videos
Open Access
Abstract
This paper presents a comprehensive approach to detect violent events in videos by combining CrimeNet, a Vision Transformer (ViT) model with structured neural learning and adversarial regularization, with an adaptive threshold sliding window model based on the Transformer architecture. CrimeNet demonstrates exceptional performance on all datasets (XD-Violence, UCF-Crime, NTU-CCTV Fights, UBI-Fights, Real Life Violence Situations, MediEval, RWF-2000, Hockey Fights, Violent Flows, Surveillance Camera Fights, and Movies Fight), achieving high AUC ROC and AUC PR values (up to 99% and 100%, respectively). However, the generalization of CrimeNet to cross-dataset experiments posed some problems, resulting in a 20–30% decrease in performance, for instance, training in UCF-Crime and testing in XD-Violence resulted in 70.20% in AUC ROC. The sliding window model with adaptive thresholding effectively solves these problems by automatically adjusting the violence detection threshold, resulting in a substantial improvement in detection accuracy. By applying the sliding window model as post-processing to CrimeNet results, we were able to improve detection accuracy by 10% to 15% in cross-dataset experiments. Future lines of research include improving generalization, addressing data imbalance, exploring multimodal representations, testing in real-world applications, and extending the approach to complex human interactions.Keywords
Funding Information
- Ministerio de Ciencia e Innovación (PID2021-126359OB-I00)
This publication has 39 references indexed in Scilit:
- CrimeNet: Neural Structured Learning using Vision Transformer for violence detectionNeural Networks, 2023
- An overview of violence detection techniques: current challenges and future directionsArtificial Intelligence Review, 2022
- Semantic multimodal violence detection based on local-to-global embeddingNeurocomputing, 2022
- Deep Learning for Automatic Violence Detection: Tests on the AIRTLab DatasetIEEE Access, 2021
- Efficient Spatio-Temporal Modeling Methods for Real-Time Violence RecognitionIEEE Access, 2021
- A dataset for automatic violence detection in videosData in Brief, 2020
- Violence Detection in Videos by Combining 3D Convolutional Neural Networks and Support Vector MachinesApplied Artificial Intelligence, 2020
- Reduce False Positive Alerts for Elderly Person Fall Video-Detection Algorithm by convolutional neural network modelProcedia Computer Science, 2019
- Breaking down violence detection: Combining divide-et-impera and coarse-to-fine strategiesNeurocomputing, 2016
- Reduce false positives for object detection by a priori probability in videosNeurocomputing, 2016