ClipGrader: Leveraging Vision-Language Models for Robust Label Quality Assessment in Object Detection
Preprint
Abstract
High-quality annotations are essential for object detection models, but ensuring label accuracy - especially for bounding boxes - remains both challenging and costly. This paper introduces ClipGrader, a novel approach that leverages vision-language models to automatically assess the accuracy of bounding box annotations. By adapting CLIP (Contrastive Language-Image Pre-training) to evaluate both class label correctness and spatial precision of bounding box, ClipGrader offers an effective solution for grading object detection labels. Tested on modified object detection datasets with artificially disturbed bounding boxes, ClipGrader achieves 91% accuracy on COCO with a 1.8% false positive rate. Moreover, it maintains 87% accuracy with a 2.1% false positive rate when trained on just 10% of the COCO data. ClipGrader also scales effectively to larger datasets such as LVIS, achieving 79% accuracy across 1,203 classes. Our experiments demonstrate ClipGrader's ability to identify errors in existing COCO annotations, highlighting its potential for dataset refinement. When integrated into a semi-supervised object detection (SSOD) model, ClipGrader readily improves the pseudo label quality, helping achieve higher mAP (mean Average Precision) throughout the training process. ClipGrader thus provides a scalable AI-assisted tool for enhancing annotation quality control and verifying annotations in large-scale object detection datasets.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks2024
- A Survey on Semi-Automated and Automated Approaches for Video Annotation2022
- Grounded Language-Image Pre-training2022
- Image Segmentation Using Text and Image Prompts2022
- RegionCLIP: Region-based Language-Image Pretraining2022
- The Effect of Improving Annotation Quality on Object Detection Datasets: A Preliminary Study2022
- End-to-End Semi-Supervised Object Detection with Soft Teacher2021
- LVIS: A Dataset for Large Vocabulary Instance Segmentation2019
- Visual question answering: A survey of methods and datasetsComputer Vision and Image Understanding, 2017
- Efficiently Scaling up Crowdsourced Video AnnotationInternational Journal of Computer Vision, 2012