Where and What: Contextual Dynamics-Aware Anomaly Detection in Surveillance Videos

Authors:
Publisher website
Google Scholar
Abstract
In surveillance environments, detecting anomalies requires understanding the contextual dynamics of the environment, human behaviors, and movements within a scene. Effective anomaly detection must address both the where and what of events, but existing approaches such as unimodal action-based methods or LLM-integrated multimodal frameworks have limitations. These methods either rely on implicit scene information, making it difficult to localize where anomalies occur, or fail to adapt to surveillance specific challenges such as view changes, subtle actions, low light conditions, and crowded scenes. As a result, these challenges hinder accurate detection of what occurs. To overcome these limitations, our system takes advantage of features from a lightweight scene classification model to discern where an event occurs, acquiring explicit location-based context. To identify what events occur, it focuses on atomic actions, which remain underexplored in this field and are better suited to interpreting intricate abnormal behaviors than conventional abstract action features. To achieve robust anomaly detection, the proposed Temporal-Semantic Relationship Network (TSRN) models spatio-temporal relationships among multimodal features and employs a Segment-selective Focal Margin loss (SFML) to effectively address class imbalance, outperforming conventional MIL-based methods. Compared to existing methods, experimental results demonstrate that our system significantly reduces false alarms while maintaining robustness across diverse scenarios. Quantitative and qualitative evaluations on public datasets validate the practical effectiveness of the proposed method for real-world surveillance applications.
Funding Information
  • Development of a Mobile-Platform-Based Solution for Proactive Safety Services through the Korea Institutes of Police Technology (RS-2025-25393280)
  • MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP- 2025-RS-2022-00156225)
  • Korea National Police Agency and the Ministry of the Interior and Safety (RS- 2024-00405100)
  • National Research Foundation of Korea (NRF) grant (RS-2024-00415812)
  • Korea Institute of Science and Technology (KIST) Institutional Program (Project No.2E33612)