FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localization
Open Access
- 28 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2041-2049
Abstract
Zero-shot anomaly detection (ZSAD) methods detect anomalies without prior access to known normal or abnormal samples within target categories. Existing methods typically rely on pretrained multimodal models, computing similarities between manually crafted textual features representing ''normal'' or ''abnormal'' semantics and image patch features to detect anomalies. However, the generic descriptions of ''abnormal'' often fail to precisely match diverse types of anomalies across different object categories. Additionally, computing feature similarities for single patches struggles to pinpoint specific locations of anomalies with various sizes and scales. To address these issues, we propose a novel ZSAD method called FiLo, comprising two components: adaptively learned Fine-Grained Description (FG-Des) and position-enhanced High-Quality Localization (HQ-Loc). FG-Des introduces fine-grained anomaly descriptions for each category using Large Language Models (LLMs) and employs adaptively learned textual templates to enhance the accuracy and interpretability of anomaly detection. HQ-Loc, utilizing Grounding DINO for preliminary localization, position-enhanced text prompts, and Multi-scale Multi-shape Cross-modal Interaction (MMCI) module, facilitates more accurate localization of anomalies of different sizes and shapes. Experimental results on datasets like MVTec and VisA demonstrate that FiLo significantly improves the performance of ZSAD in both detection and localization, achieving state-of-the-art performance with an image-level AUC of 83.9% and a pixel-level AUC of 95.9% on the VisA dataset. Code is available at https://github.com/CASIA-IVA-Lab/FiLo.Keywords
All Related Versions
Funding Information
- National Key Research and Development Program of China (2023ZD0120400)
- National Natural Science Foundation of China (62276260, 62176254, 62076235)
This publication has 19 references indexed in Scilit:
- SPot-the-Difference Self-supervised Pre-training for Anomaly Detection and Segmentation2022
- Learning to Prompt for Vision-Language ModelsInternational Journal of Computer Vision, 2022
- Towards Total Recall in Industrial Anomaly Detection2022
- Anomaly Detection via Reverse Distillation from One-Class Embedding2022
- RepVGG: Making VGG-style ConvNets Great Again2021
- PaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization2021
- MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection2019
- Sigmoid-weighted linear units for neural network function approximation in reinforcement learningNeural Networks, 2018
- Going deeper with convolutions2015
- ImageNet: A large-scale hierarchical image database2009