Theoretical analysis and experimental validation of volume bias of soft Dice optimized segmentation maps in the context of inherent uncertainty
Preprint
Abstract
The clinical interest is often to measure the volume of a structure, which is typically derived from a segmentation. In order to evaluate and compare segmentation methods, the similarity between a segmentation and a predefined ground truth is measured using popular discrete metrics, such as the Dice score. Recent segmentation methods use a differentiable surrogate metric, such as soft Dice, as part of the loss function during the learning phase. In this work, we first briefly describe how to derive volume estimates from a segmentation that is, potentially, inherently uncertain or ambiguous. This is followed by a theoretical analysis and an experimental validation linking the inherent uncertainty to common loss functions for training CNNs, namely cross-entropy and soft Dice. We find that, even though soft Dice optimization leads to an improved performance with respect to the Dice score and other measures, it may introduce a volume bias for tasks with high inherent uncertainty. These findings indicate some of the method's clinical limitations and suggest doing a closer ad-hoc volume analysis with an optional re-calibration step.Keywords
All Related Versions
- Version 1, 2022-11-08, ArXiv
- Published version: Medical Image Analysis, 67, 101833.
This publication has 18 references indexed in Scilit:
- Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methodsMachine Learning, 2021
- Optimization for Medical Image Segmentation: Theory and Practice When Evaluating With Dice Score or Jaccard IndexIEEE Transactions on Medical Imaging, 2020
- Optimization with Soft Dice Can Lead to a Volumetric Bias2020
- PHiSeg: Capturing Uncertainty in Medical Image Segmentation2019
- Assessing Reliability and Challenges of Uncertainty Estimations for Medical Image Segmentation2019
- Optimizing the Dice Score and Jaccard Index for Medical Image Segmentation: Theory and Practice2019
- Contra-Lateral Information CNN for Core Lesion Segmentation Based on Native CTP in Acute Stroke2019
- Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic featuresScientific Data, 2017
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Morphometric analysis of white matter lesions in MR images: method and validationIEEE Transactions on Medical Imaging, 1994