Probabilistic Object Detection: Definition and Evaluation
- 1 March 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1020-1029
Abstract
We introduce Probabilistic Object Detection, the task of detecting objects in images and accurately quantifying the spatial and semantic uncertainties of the detections. Given the lack of methods capable of assessing such probabilistic object detections, we present the new Probability-based Detection Quality measure (PDQ). Unlike AP-based measures, PDQ has no arbitrary thresholds and rewards spatial and label quality, and foreground/background separation quality while explicitly penalising false positive and false negative detections. We contrast PDQ with existing mAP and moLRP measures by evaluating state-of-the-art detectors and a Bayesian object detector based on Monte Carlo Dropout. Our experiments indicate that conventional object detectors tend to be spatially overconfident and thus perform poorly on the task of probabilistic object detection. Our paper aims to encourage the development of new object detection approaches that provide detections with accurately estimated spatial and label uncertainties and are of critical importance for deployment on robots and embodied AI systems in the real world.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Deep learning for detecting robotic graspsThe International Journal of Robotics Research, 2015
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- What is a good evaluation measure for semantic segmentation?2013
- Safe and Interpretable Machine Learning: A Methodological Review2013
- Multi-column deep neural network for traffic sign classificationNeural Networks, 2012
- Diagnosing Error in Object Detectors2012
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- Robust Real-Time Face DetectionInternational Journal of Computer Vision, 2004
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955