Probability-based Detection Quality (PDQ): A Probabilistic Approach to Detection Evaluation
Preprint
Abstract
We propose a new visual object detector evaluation measure which not only assesses detection quality, but also accounts for the spatial and label uncertainties produced by object detection systems. Current evaluation measures such as mean average precision (mAP) do not take these two aspects into account, accepting detections with no spatial uncertainty and using only the label with the winning score instead of a full class probability distribution to rank detections. To overcome these limitations, we propose the probability-based detection quality (PDQ) measure which evaluates both spatial and label probabilities, requires no thresholds to be predefined, and optimally assigns ground-truth objects to detections. Our experimental evaluation shows that PDQ rewards detections with accurate spatial probabilities and explicitly evaluates label probability to determine detection quality. PDQ aims to encourage the development of new object detection approaches that provide meaningful spatial and label uncertainty measures.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Risks of Deep Reinforcement Learning Applied to Fall Prevention Assist by Autonomous Mobile Robots in the HospitalBig Data and Cognitive Computing, 2018
- The limits and potentials of deep learning for roboticsThe International Journal of Robotics Research, 2018
- Deep learning for detecting robotic graspsThe International Journal of Robotics Research, 2015
- Multi-column deep neural network for traffic sign classificationNeural Networks, 2012
- The Hungarian Method for the Assignment Problem2009
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009