Spatial–Temporal Feature Fusion with Multi-Head Attention for Supervised Anomaly Localization
Open Access
Abstract
Localizing anomalous activities in surveillance video requires spatial appearance to be interpreted within its temporal context. This study presents a supervised dual-branch architecture that combines YOLOv8n features from a key frame with R3D-18 features from a 16-frame RGB clip. Channel projection and spatial alignment enable convolutional fusion of the two representations. Residual multi-head self-attention then models relationships among spatial positions in the fused feature map. Dense prediction heads estimate activity classes, objectness scores, and bounding boxes on the key frame. On the six categories of UCF-Crime2Local, the reference run without nonmaximum suppression (NMS) achieves 54.21% precision, 57.85% recall, and 53.44% . Across three training seeds, mean no-NMS is %. The finalized pipeline with class-aware NMS at IoU 0.5 achieves % across the same seeds. Values following ± denote sample standard deviations. The complete model exceeds the fusion-without-attention variant by 4.44 mAP percentage points under the common experimental protocol. Class-wise analysis, controlled component comparisons, repeated runs, and video-level uncertainty analysis characterize the model within the specified key-frame localization task. The accompanying implementation documents temporal sampling, annotation validation, optimization, checkpoint selection, and prediction export. Continuous-video evaluation derives event intervals from category-specific frame scores, achieving 57.78% event at temporal IoU 0.3 and a median onset delay of 0.80 s among matched events. These intervals are obtained through temporal postprocessing rather than a learned boundary-prediction head. Cross-dataset generalization and live-stream deployment remain to be evaluated.
Keywords
Funding Information
- Muhammad Naeem, Department of Computer Science and Engineering, Mälardalen University, Västerås, Sweden
This publication has 37 references indexed in Scilit:
- End-to-End WiFi CSI Sensing Framework for Accurate and Privacy-Preserving Activity Recognition2025
- EventVAD: Training-Free Event-Aware Video Anomaly Detection2025
- CSHA-CSI: Towards Contactless Sensing of Human Activities Utilizing WiFi Channel State Information2024
- YOWOv3: A Lightweight Spatio-Temporal Joint Network for Video Action DetectionIEEE Transactions on Circuits and Systems for Video Technology, 2024
- Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep ModelsACM Computing Surveys, 2024
- VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly DetectionProceedings of the AAAI Conference on Artificial Intelligence, 2024
- MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly DetectionProceedings of the AAAI Conference on Artificial Intelligence, 2023
- A Comprehensive Review on Vision-Based Violence Detection in Surveillance VideosACM Computing Surveys, 2023
- A Survey of Single-Scene Video Anomaly DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016