A formal method for selecting evaluation metrics for image segmentation
Authors:
- 1 October 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15224880,p. 932-936
Abstract
Evaluating the quality of segmentations is an important process in image processing, especially in the medical domain. Many evaluation metrics have been used in evaluating segmentation. There exists no formal way to choose the most suitable metric(s) for a particular segmentation task and/or particular data. In this paper we propose a formal method for choosing the most suitable metrics for evaluating the quality of segmentations with respect to ground truth segmentations. The proposed method depends on measuring the bias of metrics towards/against the properties of the the segmentations being evaluated. We firstly demonstrate how metrics can have bias towards/against particular properties and then we propose a general method for ranking metrics according to their overall bias. We finally demonstrate for 3D medical image segmentations that ranking produced using metrics with low overall bias strongly correlate with manual rankings done by an expert.Keywords
This publication has 11 references indexed in Scilit:
- Axiometrics2013
- A subjective experiment for 3D-mesh segmentation evaluation2010
- Information theoretic measures for clusterings comparison2009
- A comparison of extrinsic clustering evaluation metrics based on formal constraintsInformation Retrieval Journal, 2008
- On the reliability of information retrieval metrics based on graded relevanceInformation Processing & Management, 2007
- Evaluating evaluation metrics based on the bootstrap2006
- Using AUC and accuracy in evaluating learning algorithmsIEEE Transactions on Knowledge and Data Engineering, 2005
- The effect of topic set size on retrieval experiment error2002
- Evaluating evaluation measure stability2000
- On a Test of Whether one of Two Random Variables is Stochastically Larger than the OtherThe Annals of Mathematical Statistics, 1947