Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation
Open Access
- 1 June 2024
- journal article
- research article
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 8 (8) , e2300174
Abstract
PURPOSE: The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors. METHODS: Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure. RESULTS: Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.Keywords
This publication has 21 references indexed in Scilit:
- Systematic Review and Meta-analysis of the Association Between Radiation Therapy Treatment Volume and Patient OutcomesInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2023
- Comprehensive Quantitative Evaluation of Variability in Magnetic Resonance-Guided Delineation of Oropharyngeal Gross Tumor Volumes and High-Risk Clinical Target Volumes: An R-IDEAL Stage 0 Prospective StudyInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2022
- Machine Learning for Auto-Segmentation in Radiotherapy PlanningClinical Oncology, 2022
- Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical reviewRadiotherapy and Oncology, 2021
- Simple Python Module for Conversions Between DICOM Images and Radiation Therapy Structures, Masks, and Prediction ArraysPractical Radiation Oncology, 2021
- A Systematic Review of Contouring Guidelines in Radiation Oncology: Analysis of Frequency, Methodology, and Delivery of Consensus RecommendationsInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2020
- The American Society for Radiation Oncology 2017 Radiation Oncologist Workforce StudyInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2019
- REDCap: web-based software for all types of data storage and collectionSpinal Cord, 2018
- Interobserver variability in radiation therapy plan output: Results of a single-institution studyPractical Radiation Oncology, 2016
- Impact of facility volume on outcomes in patients with squamous cell carcinoma of the anal canal: Analysis of the National Cancer Data BaseCancer, 2016