Multi‐site quality and variability analysis of 3D FDG PET segmentations based on phantom and clinical image data
Authors:
- Reinhard R. BeichelRBReinhard R. Beichel
- Christian BauerCBChristian Bauer
- Ethan J. UlrichEUEthan J. Ulrich
- Payam AhmadvandPAPayam Ahmadvand
- Mikalai M. BudzevichMBMikalai M. Budzevich
- Robert J. GilliesRGRobert J. Gillies
- Dmitry GoldgofDGDmitry Goldgof
- Milan GrkovskiMGMilan Grkovski
- Ghassan HamarnehGHGhassan Hamarneh
- Qiao HuangQHQiao Huang
- Paul E. KinahanPKPaul E. Kinahan
Open Access
- 10 December 2016
- journal article
- research article
- Published by Wiley in Medical Physics
- Vol. 44 (2) , 479-496
Abstract
Purpose: Radiomics utilizes a large number of image‐derived features for quantifying tumor characteristics that can in turn be correlated with response and prognosis. Unfortunately, extraction and analysis of such image‐based features is subject to measurement variability and bias. The challenge for radiomics is particularly acute in Positron Emission Tomography (PET) where limited resolution, a high noise component related to the limited stochastic nature of the raw data, and the wide variety of reconstruction options confound quantitative feature metrics. Extracted feature quality is also affected by tumor segmentation methods used to define regions over which to calculate features, making it challenging to produce consistent radiomics analysis results across multiple institutions that use different segmentation algorithms in their PET image analysis. Understanding each element contributing to these inconsistencies in quantitative image feature and metric generation is paramount for ultimate utilization of these methods in multi‐institutional trials and clinical oncology decision making.Methods: To assess segmentation quality and consistency at the multi‐institutional level, we conducted a study of seven institutional members of the National Cancer Institute Quantitative Imaging Network. For the study, members were asked to segment a common set of phantom PET scans acquired over a range of imaging conditions as well as a second set of head and neck cancer (HNC) PET scans. Segmentations were generated at each institution using their preferred approach. In addition, participants were asked to repeat segmentations with a time interval between initial and repeat segmentation. This procedure resulted in overall 806 phantom insert and 641 lesion segmentations. Subsequently, the volume was computed from the segmentations and compared to the corresponding reference volume by means of statistical analysis.Results: On the two test sets (phantom and HNC PET scans), the performance of the seven segmentation approaches was as follows. On the phantom test set, the mean relative volume errors ranged from 29.9 to 87.8% of the ground truth reference volumes, and the repeat difference for each institution ranged between −36.4 to 39.9%. On the HNC test set, the mean relative volume error ranged between −50.5 to 701.5%, and the repeat difference for each institution ranged between −37.7 to 31.5%. In addition, performance measures per phantom insert/lesion size categories are given in the paper. On phantom data, regression analysis resulted in coefficient of variation (CV) components of 42.5% for scanners, 26.8% for institutional approaches, 21.1% for repeated segmentations, 14.3% for relative contrasts, 5.3% for count statistics (acquisition times), and 0.0% for repeated scans. Analysis showed that the CV components for approaches and repeated segmentations were significantly larger on the HNC test set with increases by 112.7% and 102.4%, respectively.Conclusion: Analysis results underline the importance of PET scanner reconstruction harmonization and imaging protocol standardization for quantification of lesion volumes. In addition, to enable a distributed multi‐site analysis of FDG PET images, harmonization of analysis approaches and operator training in combination with highly automated segmentation methods seems to be advisable. Future work will focus on quantifying the impact of segmentation variation on radiomics system performance.Keywords
This publication has 30 references indexed in Scilit:
- Use of wall-less18F-doped gelatin phantoms for improved volume delineation and quantification in PET/CTPhysics in Medicine & Biology, 2014
- The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information RepositoryJournal of Digital Imaging, 2013
- Influence of cold walls on PET image quantification and volume segmentation: A phantom studyMedical Physics, 2013
- Harmonizing SUVs in multicentre trials when using different generation PET systems: prospective validation in non-small cell lung cancer patientsEuropean Journal of Nuclear Medicine and Molecular Imaging, 2013
- Radiomics: the process and the challengesMagnetic Resonance Imaging, 2012
- A gel tumour phantom for assessment of the accuracy of manual and automatic delineation of gross tumour volume from FDG-PET/CTActa Oncologica, 2012
- 18F-FDG Metabolic Tumor Volume and Total Glycolytic Activity of Oral Cavity and Oropharyngeal Squamous Cell Cancer: Adding Value to Clinical StagingJournal of Nuclear Medicine, 2012
- SUVref: reducing reconstruction-dependent variation in PET SUVEJNMMI Research, 2011
- Effects of cold sphere walls in PET phantom measurements on the volume reproducing thresholdPhysics in Medicine & Biology, 2010
- New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1)European Journal of Cancer, 2009