Human-Centric Anomaly Detection in Surveillance Videos Using YOLO-World and Spatio-Temporal Deep Learning
Preprint
Abstract
Anomaly detection in surveillance videos remains a challenging task due to the diversity of abnormal events, class imbalance, and scene-dependent visual clutter. To address these issues, we propose a robust deep learning framework that integrates human-centric preprocessing with spatio-temporal modeling for multi-class anomaly classification. Our pipeline begins by applying YOLO-World - an open-vocabulary vision-language detector - to identify human instances in raw video clips, followed by ByteTrack for consistent identity-aware tracking. Background regions outside detected bounding boxes are suppressed via Gaussian blurring, effectively reducing scene-specific distractions and focusing the model on behaviorally relevant foreground content. The refined frames are then processed by an ImageNet-pretrained InceptionV3 network for spatial feature extraction, and temporal dynamics are captured using a bidirectional LSTM (BiLSTM) for sequence-level classification. Evaluated on a five-class subset of the UCF-Crime dataset (Normal, Burglary, Fighting, Arson, Explosion), our method achieves a mean test accuracy of 92.41% across three independent trials, with per-class F1-scores consistently exceeding 0.85. Comprehensive evaluation metrics - including confusion matrices, ROC curves, and macro/weighted averages - demonstrate strong generalization and resilience to class imbalance. The results confirm that foreground-focused preprocessing significantly enhances anomaly discrimination in real-world surveillance scenarios.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Deep BiLSTM Attention Model for Spatial and Temporal Anomaly Detection in Video SurveillanceSensors, 2025
- A Scalable and Generalised Deep Learning Framework for Anomaly Detection in Surveillance VideosInternational Journal of Intelligent Systems, 2025
- Shoplifting Detection Using Hybrid Neural Network CNN-BiLSMT and Development of Benchmark DatasetApplied Sciences, 2023
- A CNN-RNN Combined Structure for Real-World Violence Detection in Surveillance CamerasApplied Sciences, 2022
- An Efficient Anomaly Recognition Framework Using an Attention Residual LSTM in Surveillance VideosSensors, 2021
- Review of deep learning: concepts, CNN architectures, challenges, applications, future directionsJournal of Big Data, 2021
- LSTM Fully Convolutional Networks for Time Series ClassificationIEEE Access, 2017
- A Survey on Visual Surveillance of Object Motion and BehaviorsIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2004