Demonstration of VCR: A Tabular Data Slicing Approach to Understanding Object Detection Model Performance
Open Access
- 1 August 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the VLDB Endowment
- Vol. 17 (12) , 4453-4456
Abstract
In this demonstration, we present VCR, an automated slice discovery method (SDM) for object detection models that helps practitioners identify and explain specific scenarios in which their models exhibit systematic errors. VCR leverages the capabilities of vision foundation models to generate segment-level visual concepts that serve as interpretable explanation primitives. By integrating these visual concepts with additional image metadata in a tabular format, VCR uses a scalable frequent itemset mining-based technique to identify common patterns associated with model performance. We will demonstrate VCR's capabilities through three usage scenarios. First, users can explore the automatically extracted visual concepts and their associated labels. Second, users can run slice finding on a large object detection dataset and visually inspect the results to discover systematic errors. Finally, users can iteratively refine their slicing results by providing feedback on the granularity of visual concepts and the quality of the generated labels. These scenarios will illustrate how VCR can aid practitioners in discovering non-trivial gaps in their models' performance, providing actionable insights for model improvement.Keywords
This publication has 7 references indexed in Scilit:
- Segment Anything2023
- POEM: Pattern-Oriented Explanations of Convolutional Neural NetworksProceedings of the VLDB Endowment, 2023
- MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining2023
- The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models2022
- Looking for Trouble: Analyzing Classifier Behavior via Pattern Divergence2021
- BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning2020
- Silhouettes: A graphical aid to the interpretation and validation of cluster analysisJournal of Computational and Applied Mathematics, 1987