A Language Vision Model Approach for Automated Tumor Contouring in Radiation Oncology
Open Access
- 31 July 2025
- journal article
- research article
- Published by MDPI AG in Bioengineering
- Vol. 12 (8) , 835
Abstract
Background: Lung cancer ranks as the leading cause of cancer-related mortality worldwide. The complexity of tumor delineation, crucial for radiation therapy, requires expertise often unavailable in resource-limited settings. Artificial Intelligence (AI), particularly with advancements in deep learning (DL) and natural language processing (NLP), offers potential solutions yet is challenged by high false positive rates. Purpose: The Oncology Contouring Copilot (OCC) system is developed to leverage oncologist expertise for precise tumor contouring using textual descriptions, aiming to increase the efficiency of oncological workflows by combining the strengths of AI with human oversight. Methods: Our OCC system initially identifies nodule candidates from CT scans. Employing Language Vision Models (LVMs) like GPT-4V, OCC then effectively reduces false positives with clinical descriptive texts, merging textual and visual data to automate tumor delineation, designed to elevate the quality of oncology care by incorporating knowledge from experienced domain experts. Results: The deployment of the OCC system resulted in a 35.0% reduction in the false discovery rate, a 72.4% decrease in false positives per scan, and an F1-score of 0.652 across our dataset for unbiased evaluation. Conclusions: OCC represents a significant advance in oncology care, particularly through the use of the latest LVMs, improving contouring results by (1) streamlining oncology treatment workflows by optimizing tumor delineation and reducing manual processes; (2) offering a scalable and intuitive framework to reduce false positives in radiotherapy planning using LVMs; (3) introducing novel medical language vision prompt techniques to minimize LVM hallucinations with ablation study; and (4) conducting a comparative analysis of LVMs, highlighting their potential in addressing medical language vision challenges.Keywords
All Related Versions
Funding Information
- National Cancer Institute of the National Institutes of Health (R25CA288263, R37CA229417)
This publication has 28 references indexed in Scilit:
- Advancing the frontier of artificial intelligence on emerging technologies to redefine cancer diagnosis and careComputers in Biology and Medicine, 2025
- Technical note: Generalizable and promptable artificial intelligence model to augment clinical delineation in radiation oncologyMedical Physics, 2024
- The global burden of lung cancer: current status and future trendsNature Reviews Clinical Oncology, 2023
- Controlling False-Positives in Automatic Lung Nodule Detection by Adding 3D Cuboid Attention to a Convolutional Neural NetworkBiomedical Signal Processing and Control, 2023
- An attentive and adaptive 3D CNN for automatic pulmonary nodule detection in CT imageExpert Systems with Applications, 2022
- Biomarkers for immune checkpoint inhibition in non–small cell lung cancer (NSCLC)Cancer, 2019
- Automated pulmonary nodule detection in CT images using deep convolutional neural networksPattern Recognition, 2019
- Multilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule DetectionIEEE Transactions on Biomedical Engineering, 2016
- Closing the Cancer Divide Through Ubuntu: Information and Communication Technology-Powered Models for Global Radiation OncologyInternational Journal of Radiation Oncology - Biology - Physics (IJROBP), 2016
- Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012International Journal of Cancer, 2014