Real-world performance evaluation of commercial autocontouring software for head and neck cancer radiotherapy
Open Access
- 8 July 2025
- journal article
- Published by Oxford University Press (OUP) in British Journal of Radiology
- Vol. 98 (1174) , 1632-1641
Abstract
Objectives: To comprehensively evaluate the ability of ART-Plan (autocontouring software using artificial intelligence [AI]) to contour head and neck cancer (HNC) radiotherapy structures (organs-at-risk and elective nodal volumes) in a real-world setting, as required by NICE guidelines. Methods: Retrospective evaluation (n = 60) compared clinically used contours to AI-contours, using volumetric dice similarity coefficient (VDSC) and blinded radiation oncologist (RO) contour preference assessment. AI-contours were then generated prospectively for HNC radiotherapy patients (n = 61), before review by ROs. ROs recorded qualitative scoring and review/editing time. Geometric and dosimetric comparison of AI-contours and final contours was undertaken. Correlation coefficients between all metrics were calculated. Results: Retrospective median VDSC varied widely for different structures (0.23-0.88). 31.4% blinded contour assessments preferred clinician-generated contours, 32.9% preferred AI-generated contours, 35.7% saw no significant difference. Prospective evaluation showed AI-contour yielded significant time-saving in reviewing/editing for all structures compared to manual contouring. Qualitative scores demonstrated most structures had median scoring ≥4 (indicating no editing required). Geometric metrics showed high similarity for all structures except larynx. Dosimetric evaluation demonstrated clinically significant dose differences for larynx and elective nodal volumes. Strong correlation was seen between qualitative scoring and all geometric metrics. Conclusions: AI-contours showed excellent qualitative performance and facilitated time-saving. Protocol differences exist between commercial AI-solutions and implementing centres. Ongoing final clinician review of AI-contours remains essential. Advances in knowledge: This is the first study demonstrating ART-Plan’s capability for excellent qualitative ratings and significant time-savings for HNC radiotherapy contouring in a real-world workflow setting. 5-point Likert-scale qualitative scoring correlates strongly with geometric metrics.Keywords
Funding Information
- Radiation Research Unit at the Cancer Research UK City of London Centre (C7893/A28990)
This publication has 24 references indexed in Scilit:
- Structure and Processes of Existing Practice in Radiotherapy Peer Review: A Systematic Review of the LiteratureClinical Oncology, 2021
- Clinical implementation of deep learning contour autosegmentation for prostate radiotherapyRadiotherapy and Oncology, 2021
- Generating High-Quality Lymph Node Clinical Target Volumes for Head and Neck Cancer Radiation Therapy Using a Fully Automated Deep Learning-Based ApproachInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2020
- Guy’s cancer cohort – real world evidence for cancer pathwaysBMC Cancer, 2020
- Evaluation of measures for assessing time-saving of automatic organ-at-risk segmentation in radiotherapyPhysics and Imaging in Radiation Oncology, 2020
- Benefits of deep learning for delineation of organs at risk in head and neck cancerRadiotherapy and Oncology, 2019
- Rapid advances in auto-segmentation of organs at risk and target volumes in head and neck cancerRadiotherapy and Oncology, 2019
- Uncertainties in target volume delineation in radiotherapy – are they relevant and what can we do about them?Radiology and Oncology, 2016
- Systematic evaluation of three different commercial software solutions for automatic segmentation for adaptive therapy in head-and-neck, prostate and pleural cancerRadiation Oncology, 2012
- Dosimetric Predictors of Laryngeal EdemaInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2007