Cloud-Based Evaluation of Anatomical Structure Segmentation and Landmark Detection Algorithms: VISCERAL Anatomy Benchmarks
Authors:
- Oscar Jimenez-Del-ToroOJOscar Jimenez-Del-Toro
- Henning MullerHMHenning Muller
- Markus KrennMKMarkus Krenn
- Katharina GruenbergKGKatharina Gruenberg
- Abdel Aziz TahaATAbdel Aziz Taha
- Marianne WintersteinMWMarianne Winterstein
- Ivan EggelIEIvan Eggel
- Antonio Foncubierta-RodriguezAFAntonio Foncubierta-Rodriguez
- Orcun GokselOGOrcun Goksel
- Andras JakabAJAndras Jakab
- Georgios KontokotsiosGKGeorgios Kontokotsios
- Georg LangsGLGeorg Langs
Top Cited Papers
- 1 November 2016
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 35 (11) , 2459-2475
Abstract
Variations in the shape and appearance of anatomical structures in medical images are often relevant radiological signs of disease. Automatic tools can help automate parts of this manual process. A cloud-based evaluation framework is presented in this paper including results of benchmarking current state-of-the-art medical imaging algorithms for anatomical structure segmentation and landmark detection: the VISCERAL Anatomy benchmarks. The algorithms are implemented in virtual machines in the cloud where participants can only access the training data and can be run privately by the benchmark administrators to objectively compare their performance in an unseen common test set. Overall, 120 computed tomography and magnetic resonance patient volumes were manually annotated to create a standard Gold Corpus containing a total of 1295 structures and 1760 landmarks. Ten participants contributed with automatic algorithms for the organ segmentation task, and three for the landmark localization task. Different algorithms obtained the best scores in the four available imaging modalities and for subsets of anatomical structures. The annotation framework, resulting data set, evaluation setup, results and performance analysis from the three VISCERAL Anatomy benchmarks are presented in this article. Both the VISCERAL data set and Silver Corpus generated with the fusion of the participant algorithms on a larger set of non-manually-annotated medical images are available to the research community.Keywords
This publication has 40 references indexed in Scilit:
- Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: The ANODE09 studyMedical Image Analysis, 2010
- Statistical shape models for 3D medical image segmentation: A reviewMedical Image Analysis, 2009
- Online resource for validation of brain segmentation methodsNeuroImage, 2009
- Comparison of Prostate Volume Measured by Transrectal Ultrasonography and MRI with the Actual Prostate Volume Measured after Radical ProstatectomyUrologia Internationalis, 2008
- Future CAD in multi-dimensional medical imagesComputerized Medical Imaging and Graphics, 2007
- Classifier Technology and the Illusion of ProgressStatistical Science, 2006
- Current status and future potential of computer-aided diagnosis in medical imagingBritish Journal of Radiology, 2005
- Automated model-based organ delineation for radiotherapy planning in prostatic regionInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2004
- Simultaneous Truth and Performance Level Estimation (STAPLE): An Algorithm for the Validation of Image SegmentationIEEE Transactions on Medical Imaging, 2004
- Measures of the Amount of Ecologic Association Between SpeciesEcology, 1945