PromptVAD: Abnormal Prompt via Vision-Language Model

Authors:
Publisher website
Google Scholar
Abstract
Weakly supervised video anomaly detection (WSVAD) aims at predicting frame-level anomaly scores by modeling training videos with video-level annotations. The category names of abnormal events contain high-level knowledge abstracted by humans about abnormalities, which is of great help in identifying abnormal events. To utilize the knowledge implicit in category names, based on the visual-language pretraining model, we introduce a learnable abnormal prompt from three aspects: learnable domain prompt, learnable category prompt, and nonlearnable category definition prompt. Based on the learnable abnormal prompt, we propose a novel fine-grained WSVAD method: PromptVAD, which exploits a learnable abnormal prompt to reduce the semantic gap between visual images and anomaly categories. Through a similarity measure and our proposed coarse-grained two-class prompt module, our PromptVAD jointly learns coarse-grained and fine-grained VAD. Extensive experimental results on the ShanghaiTech, University of Central Florida (UCF)-Crime, and XD-Violence datasets show that our method achieves state-of-the-art performance. Specifically, our method achieves an area under the curve (AUC) of 88.62% on the UCF-Crime dataset.
Funding Information
  • National Natural Science Foundation of China (62406231)
  • National Natural Science Foundation of China (62576264)
  • Shaanxi Province Post-Doctoral Research Project Funding (2024BSHSDZZ119)
  • China Post-Doctoral Science Foundation (2023M742738)
  • National Science and Technology Major Project (2025ZD0551500)
  • National Science and Technology Major Project (2025ZD0551502)
  • Joint Fund Project of National Natural Science Foundation of China (U22B2054)
  • Fund for Foreign Scholars in University Research and Teaching Programs (B07048)
  • Program for Cheung Kong Scholars and Innovative Research Team in University (IRT_15R53)
  • Key Scientific Technological Innovation Research Project by Ministry of Education
  • National Key Laboratory of Human–Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University (HMHAI-202404)
  • National Key Laboratory of Human–Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University (HMHAI-202405)

This publication has 80 references indexed in Scilit: