Ernst Th Scholten
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Ernst Th Scholten has a total of 161 co-authors. Here are the top co-authors by number of shared publications:
- Colin Jacobs
From the Department of Medical Imaging, Radboud Institute of Medical Innovation, Radboud University Medical Center, Nijmegen, the Netherlands
11 shared publications - Bram van Ginneken
Radboud University Medical Center
10 shared publications - Mathias Prokop
Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
9 shared publications - Cornelia Schaefer-Prokop
Diagnostic Image Analysis Group, Department of Medical Imaging, Radboudumc, Radboud University Medical Center, Route 767, Room 2.30, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, the Netherlands
6 shared publications - Kiran Vaidhya Venkadesh
Diagnostic Image Analysis Group (DIAG), Department of Radiology and Nuclear Medicine, Radboud UMC, Nijmegen, The Netherlands
5 shared publications - Mario Silva
Department of Medicine and Surgery (DiMeC), Scienze Radiologiche, University of Parma, Parma, Italy
4 shared publications
Latest Publications
- Open Access
Large Language Models to Label CT Radiology Reports for Incidental Findings Found in Lung Cancer Screening: A Comparative Analysis
Objective: To compare LLM and prompting strategies for labeling Specific Incidental Findings (SIFs) occurring during lung cancer screening, in radiology reports. Materials and Methods: In this retrospective study, 300 CT reports from a Dutch academic hospital were included between 19 th March 2003 and 28 th December 2020. GPT-5, GPT-4-Turbo, Llama-3.1-8B-INST, MedGemma-4B-IT, a keyword-based search method, and a radiologist labelled the reports for the presence or absence of nine SIFs. The radiologist’s scores were the reference standard. Each LLM was evaluated with four different prompting strategies: two without Chain-of-Thought (CoT), and two with CoT (one zero shot, one few shot). The automated labelling methods were evaluated against the radiologist primarily using Cohen’s Kappa, and additionally sensitivity, specificity, and F1 scores. Results: GPT-5 achieved substantial or almost perfect agreement for seven out of nine SIFs: aortic valve disease, bronchiectasis, consolidation, coronary artery calcification, emphysema, abnormality in mediastinal lymph nodes, and thyroid abnormalities, with Cohen κ values of 0.81 (0.70, 0.93), 0.97 (0.93, 1.00), 0.71 (0.62, 0.82), 0.92 (0.87, 0.96), 0.91 (0.84, 0.97), 0.71 (0.62, 0.80), and 0.92 (0.86, 0.98) respectively. For the remaining two SIFs, moderate LLM agreement was obtained with MedGemma-4B-IT and with GPT-4-Turbo with Cohen κ values of 0.48 (0.40, 0.60), 0.42 (0.31, 0.58) for interstitial lung abnormalities and mediastinal mass respectively. The highest agreement was obtained with the non-CoT prompt strategies for most SIFs. Conclusion: This study demonstrated that LLMs can automatically label radiology reports with agreement ranging from almost perfect to moderate, but not one LLM – prompt combination achieved the highest agreement across all SIFs.
Open AccessLarge Language Models to Label CT Radiology Reports for Incidental Findings Found in Lung Cancer Screening: A Comparative Analysis
Objective: To compare LLM and prompting strategies for labeling Specific Incidental Findings (SIFs) occurring during lung cancer screening, in radiology reports. Materials and Methods: In this retrospective study, 300 CT reports from a Dutch academic hospital were included between 19 th March 2003 and 28 th December 2020. GPT-5, GPT-4-Turbo, Llama-3.1-8B-INST, MedGemma-4B-IT, a keyword-based search method, and a radiologist labelled the reports for the presence or absence of nine SIFs. The radiologist’s scores were the reference standard. Each LLM was evaluated with four different prompting strategies: two without Chain-of-Thought (CoT), and two with CoT (one zero shot, one few shot). The automated labelling methods were evaluated against the radiologist primarily using Cohen’s Kappa, and additionally sensitivity, specificity, and F1 scores. Results: GPT-5 achieved substantial or almost perfect agreement for seven out of nine SIFs: aortic valve disease, bronchiectasis, consolidation, coronary artery calcification, emphysema, abnormality in mediastinal lymph nodes, and thyroid abnormalities, with Cohen κ values of 0.81 (0.70, 0.93), 0.97 (0.93, 1.00), 0.71 (0.62, 0.82), 0.92 (0.87, 0.96), 0.91 (0.84, 0.97), 0.71 (0.62, 0.80), and 0.92 (0.86, 0.98) respectively. For the remaining two SIFs, moderate LLM agreement was obtained with MedGemma-4B-IT and with GPT-4-Turbo with Cohen κ values of 0.48 (0.40, 0.60), 0.42 (0.31, 0.58) for interstitial lung abnormalities and mediastinal mass respectively. The highest agreement was obtained with the non-CoT prompt strategies for most SIFs. Conclusion: This study demonstrated that LLMs can automatically label radiology reports with agreement ranging from almost perfect to moderate, but not one LLM – prompt combination achieved the highest agreement across all SIFs.
- Open Access
- 1 July 2026
- Published by Oxford University Press (OUP) in Radiology Advances
- Vol. 3 (5)
Multicenter AI-versus Expert-Assisted RECIST Target Lesion Measurements in Follow-Up Body CT of Cancer Patients
Background: Manual lesion measurements remain the standard for assessing oncologic treatment response, despite being time-consuming and prone to substantial inter-reader variability, which may lead to inconsistent response classification and subsequent variability in treatment decisions. Purpose: To evaluate the impact of an artificial intelligence (AI) system for assisted lesion measurement on reading time and measurement consistency in follow-up CT examinations using the Response Evaluation Criteria in Solid Tumors (RECIST 1.1). Methods and Materials: In this retrospective reader study, follow-up chest-abdomen-pelvis CT examinations from 212 oncology patients collected at two Dutch hospitals were assessed by 23 readers (15 radiologists, 8 residents) recruited from 11 international institutions under three conditions: unassisted, AI-assisted, and expert-assisted (using a prior radiologist’s unassisted measurement). To prevent bias related to the source of the measurements, readers were informed that all support was AI-derived. Primary outcomes were reading time to completion and inter-observer measurement variability. For each outcome, a Bayesian generalized linear mixed model was used to analyze the results. Results: AI assistance significantly reduced per-patient reading time versus unassisted reading (-36.0 s; 95% CI: -53.0, -22.1). At the lesion level, it was associated with a small increase in variability relative to the expert-derived reference standard (1.32 mm; 95% CI: 0.83, 1.91). At the patient level, AI assistance did not meaningfully affect change in the sum of longest diameters (SLD; -0.38 mm; 95% CI: -1.57, 0.72), but increased RECIST outcome agreement by 7.7% (95% CI: 2.8, 12.7) compared with unassisted reading. Expert-assisted reading yielded even higher inter-reader agreement (13.3%; 95% CI: 8.6, 18.1). Conclusion: AI assistance reduced reading time and improved patient-level RECIST agreement, despite a small increase in lesion-level measurement variability. These findings suggest that AI-assisted RECIST assessment may improve workflow and response classification consistency, while also providing a benchmark from expert-assisted reading for future AI development.
- 1 July 2026
- Published by Oxford University Press (OUP) in Radiology Advances
- Vol. 3 (5)
Open AccessMulticenter AI-versus Expert-Assisted RECIST Target Lesion Measurements in Follow-Up Body CT of Cancer Patients
Background: Manual lesion measurements remain the standard for assessing oncologic treatment response, despite being time-consuming and prone to substantial inter-reader variability, which may lead to inconsistent response classification and subsequent variability in treatment decisions. Purpose: To evaluate the impact of an artificial intelligence (AI) system for assisted lesion measurement on reading time and measurement consistency in follow-up CT examinations using the Response Evaluation Criteria in Solid Tumors (RECIST 1.1). Methods and Materials: In this retrospective reader study, follow-up chest-abdomen-pelvis CT examinations from 212 oncology patients collected at two Dutch hospitals were assessed by 23 readers (15 radiologists, 8 residents) recruited from 11 international institutions under three conditions: unassisted, AI-assisted, and expert-assisted (using a prior radiologist’s unassisted measurement). To prevent bias related to the source of the measurements, readers were informed that all support was AI-derived. Primary outcomes were reading time to completion and inter-observer measurement variability. For each outcome, a Bayesian generalized linear mixed model was used to analyze the results. Results: AI assistance significantly reduced per-patient reading time versus unassisted reading (-36.0 s; 95% CI: -53.0, -22.1). At the lesion level, it was associated with a small increase in variability relative to the expert-derived reference standard (1.32 mm; 95% CI: 0.83, 1.91). At the patient level, AI assistance did not meaningfully affect change in the sum of longest diameters (SLD; -0.38 mm; 95% CI: -1.57, 0.72), but increased RECIST outcome agreement by 7.7% (95% CI: 2.8, 12.7) compared with unassisted reading. Expert-assisted reading yielded even higher inter-reader agreement (13.3%; 95% CI: 8.6, 18.1). Conclusion: AI assistance reduced reading time and improved patient-level RECIST agreement, despite a small increase in lesion-level measurement variability. These findings suggest that AI-assisted RECIST assessment may improve workflow and response classification consistency, while also providing a benchmark from expert-assisted reading for future AI development.
- 24 June 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- p. e260179
Benchmarking of AI and Radiologists for Indeterminate Lung Nodule Malignancy Risk Estimation on Screening CT: The LUNA25 Challenge
Purpose To compare the performance of an artificial intelligence (AI) system with that of radiologists for estimating malignancy risk of indeterminate-size nodules (5–15 mm) on low-dose CT (LDCT) within a standardized and transparent evaluation framework. Materials and Methods Teams participating in the AI study had access to a public dataset of 555 malignant and 5608 benign nodules on 4069 baseline LDCT scans from the National Lung Screening Trial (NLST) to develop AI systems. External testing was performed on 156 malignant and 312 benign size-matched nodules, all of indeterminate size, from 463 baseline scans collected from three large European lung cancer screening trials between 2004 and 2018, and the best-performing AI system (based on area under the receiver operating characteristic curve [AUC]) was selected. An observer study was conducted in which radiologists assessed 300 randomly selected nodules (100 malignant, 200 benign) from the external test set. Radiologists categorized nodules as low, intermediate, or high-risk, and the ≥ intermediate-risk threshold (intermediate or high-risk) was used to define a positive test. The selected AI system was compared with radiologists on this subset using the AUC. Results The selected AI system demonstrated superior performance to the 65 radiologists’ average (AUC, 0.78 (95% CI: 0.73, 0.84) vs 0.70 (95% CI: 0.65, 0.74), P = .001). When using the ≥ intermediate-risk threshold, the AI system correctly classified 12% more malignant nodules at matched specificity and yielded 20% fewer false positives at matched sensitivity. Conclusion The selected AI system was superior to radiologists in estimating malignancy risk of indeterminate lung nodules on LDCT. ©RSNA, 2026
- 24 June 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- p. e260179
Benchmarking of AI and Radiologists for Indeterminate Lung Nodule Malignancy Risk Estimation on Screening CT: The LUNA25 Challenge
Purpose To compare the performance of an artificial intelligence (AI) system with that of radiologists for estimating malignancy risk of indeterminate-size nodules (5–15 mm) on low-dose CT (LDCT) within a standardized and transparent evaluation framework. Materials and Methods Teams participating in the AI study had access to a public dataset of 555 malignant and 5608 benign nodules on 4069 baseline LDCT scans from the National Lung Screening Trial (NLST) to develop AI systems. External testing was performed on 156 malignant and 312 benign size-matched nodules, all of indeterminate size, from 463 baseline scans collected from three large European lung cancer screening trials between 2004 and 2018, and the best-performing AI system (based on area under the receiver operating characteristic curve [AUC]) was selected. An observer study was conducted in which radiologists assessed 300 randomly selected nodules (100 malignant, 200 benign) from the external test set. Radiologists categorized nodules as low, intermediate, or high-risk, and the ≥ intermediate-risk threshold (intermediate or high-risk) was used to define a positive test. The selected AI system was compared with radiologists on this subset using the AUC. Results The selected AI system demonstrated superior performance to the 65 radiologists’ average (AUC, 0.78 (95% CI: 0.73, 0.84) vs 0.70 (95% CI: 0.65, 0.74), P = .001). When using the ≥ intermediate-risk threshold, the AI system correctly classified 12% more malignant nodules at matched specificity and yielded 20% fewer false positives at matched sensitivity. Conclusion The selected AI system was superior to radiologists in estimating malignancy risk of indeterminate lung nodules on LDCT. ©RSNA, 2026
- 1 September 2025
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 316 (3) , e250874
External Test of a Deep Learning Algorithm for Pulmonary Nodule Malignancy Risk Stratification Using European Screening Data
A deep learning algorithm accurately estimated pulmonary nodule malignancy risk using baseline data from three European trials, retrospectively reducing false-positive classifications for indeterminate nodules while preserving sensitivity for cancer detection.
- 1 September 2025
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 316 (3) , e250874
External Test of a Deep Learning Algorithm for Pulmonary Nodule Malignancy Risk Stratification Using European Screening Data
A deep learning algorithm accurately estimated pulmonary nodule malignancy risk using baseline data from three European trials, retrospectively reducing false-positive classifications for indeterminate nodules while preserving sensitivity for cancer detection.
- Open Access
- 30 December 2024
- Published by Elsevier in Computers in Biology and Medicine
- Vol. 186, 109633
Towards safe and reliable deep learning for lung nodule malignancy estimation using out-of-distribution detection
- 30 December 2024
- Published by Elsevier in Computers in Biology and Medicine
- Vol. 186, 109633
Open AccessTowards safe and reliable deep learning for lung nodule malignancy estimation using out-of-distribution detection
- 1 August 2023
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 308 (2) , e223308
Prior CT Improves Deep Learning for Malignancy Risk Estimation of Screening-detected Pulmonary Nodules
Background Prior chest CT provides valuable temporal information (eg, changes in nodule size or appearance) to accurately estimate malignancy risk. Purpose To develop a deep learning (DL) algorithm that uses a current and prior low-dose CT examination to estimate 3-year malignancy risk of pulmonary nodules. Materials and Methods In this retrospective study, the algorithm was trained using National Lung Screening Trial data (collected from 2002 to 2004), wherein patients were imaged at most 2 years apart, and evaluated with two external test sets from the Danish Lung Cancer Screening Trial (DLCST) and the Multicentric Italian Lung Detection Trial (MILD), collected in 2004-2010 and 2005-2014, respectively. Performance was evaluated using area under the receiver operating characteristic curve (AUC) on cancer-enriched subsets with size-matched benign nodules imaged 1 and 2 years apart from DLCST and MILD, respectively. The algorithm was compared with a validated DL algorithm that only processed a single CT examination and the Pan-Canadian Early Lung Cancer Detection Study (PanCan) model. Results The training set included 10 508 nodules (422 malignant) in 4902 trial participants (mean age, 64 years ± 5 [SD]; 2778 men). The size-matched external test sets included 129 nodules (43 malignant) and 126 nodules (42 malignant). The algorithm achieved AUCs of 0.91 (95% CI: 0.85, 0.97) and 0.94 (95% CI: 0.89, 0.98). It significantly outperformed the DL algorithm that only processed a single CT examination (AUC, 0.85 [95% CI: 0.78, 0.92; P = .002]; and AUC, 0.89 [95% CI: 0.84, 0.95; P = .01]) and the PanCan model (AUC, 0.64 [95% CI: 0.53, 0.74; P < .001]; and AUC, 0.63 [95% CI: 0.52, 0.74; P < .001]). Conclusion A DL algorithm using current and prior low-dose CT examinations was more effective at estimating 3-year malignancy risk of pulmonary nodules than established models that only use a single CT examination. Clinical trial registration nos. NCT00047385, NCT00496977, NCT02837809 © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Horst and Nishino in this issue.
- 1 August 2023
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 308 (2) , e223308
Prior CT Improves Deep Learning for Malignancy Risk Estimation of Screening-detected Pulmonary Nodules
Background Prior chest CT provides valuable temporal information (eg, changes in nodule size or appearance) to accurately estimate malignancy risk. Purpose To develop a deep learning (DL) algorithm that uses a current and prior low-dose CT examination to estimate 3-year malignancy risk of pulmonary nodules. Materials and Methods In this retrospective study, the algorithm was trained using National Lung Screening Trial data (collected from 2002 to 2004), wherein patients were imaged at most 2 years apart, and evaluated with two external test sets from the Danish Lung Cancer Screening Trial (DLCST) and the Multicentric Italian Lung Detection Trial (MILD), collected in 2004-2010 and 2005-2014, respectively. Performance was evaluated using area under the receiver operating characteristic curve (AUC) on cancer-enriched subsets with size-matched benign nodules imaged 1 and 2 years apart from DLCST and MILD, respectively. The algorithm was compared with a validated DL algorithm that only processed a single CT examination and the Pan-Canadian Early Lung Cancer Detection Study (PanCan) model. Results The training set included 10 508 nodules (422 malignant) in 4902 trial participants (mean age, 64 years ± 5 [SD]; 2778 men). The size-matched external test sets included 129 nodules (43 malignant) and 126 nodules (42 malignant). The algorithm achieved AUCs of 0.91 (95% CI: 0.85, 0.97) and 0.94 (95% CI: 0.89, 0.98). It significantly outperformed the DL algorithm that only processed a single CT examination (AUC, 0.85 [95% CI: 0.78, 0.92; P = .002]; and AUC, 0.89 [95% CI: 0.84, 0.95; P = .01]) and the PanCan model (AUC, 0.64 [95% CI: 0.53, 0.74; P < .001]; and AUC, 0.63 [95% CI: 0.52, 0.74; P < .001]). Conclusion A DL algorithm using current and prior low-dose CT examinations was more effective at estimating 3-year malignancy risk of pulmonary nodules than established models that only use a single CT examination. Clinical trial registration nos. NCT00047385, NCT00496977, NCT02837809 © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Horst and Nishino in this issue.
- 1 November 2021
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 3 (6)
Deep Learning for Lung Cancer Detection on Screening CT Scans: Results of a Large-Scale Public Competition and an Observer Study with 11 Radiologists
An observer study showed that two of the three top-performing algorithms from a public competition (Kaggle Data Science Bowl 2017) attained performances that were not significantly worse than that ... Purpose To determine whether deep learning algorithms developed in a public competition could identify lung cancer on low-dose CT scans with a performance similar to that of radiologists. Materials...
- 1 November 2021
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 3 (6)
Deep Learning for Lung Cancer Detection on Screening CT Scans: Results of a Large-Scale Public Competition and an Observer Study with 11 Radiologists
An observer study showed that two of the three top-performing algorithms from a public competition (Kaggle Data Science Bowl 2017) attained performances that were not significantly worse than that ... Purpose To determine whether deep learning algorithms developed in a public competition could identify lung cancer on low-dose CT scans with a performance similar to that of radiologists. Materials...
- 1 September 2021
- Published by Radiological Society of North America (RSNA) in Radiology: Imaging Cancer
- Vol. 3 (5) , e200160
Assisted versus Manual Interpretation of Low-Dose CT Scans for Lung Cancer Screening: Impact on Lung-RADS Agreement
An observer study with seven radiologists showed that using a dedicated CT lung screening viewer (with computer-aided detection, a segmentation algorithm, and automatic Lung Imaging Reporting and D... Purpose To compare the inter- and intraobserver agreement and reading times achieved when assigning Lung Imaging Reporting and Data System (Lung-RADS) categories to baseline and follow-up lung canc...
- 1 September 2021
- Published by Radiological Society of North America (RSNA) in Radiology: Imaging Cancer
- Vol. 3 (5) , e200160
Assisted versus Manual Interpretation of Low-Dose CT Scans for Lung Cancer Screening: Impact on Lung-RADS Agreement
An observer study with seven radiologists showed that using a dedicated CT lung screening viewer (with computer-aided detection, a segmentation algorithm, and automatic Lung Imaging Reporting and D... Purpose To compare the inter- and intraobserver agreement and reading times achieved when assigning Lung Imaging Reporting and Data System (Lung-RADS) categories to baseline and follow-up lung canc...
- Top Cited Papers
- 1 August 2021
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 300 (2) , 438-447
Deep Learning for Malignancy Risk Estimation of Pulmonary Nodules Detected at Low-Dose Screening CT
A deep learning algorithm, developed for malignancy risk estimation of pulmonary nodules detected at screening CT, demonstrated good discriminative performance in a large independent validation set. Background Accurate estimation of the malignancy risk of pulmonary nodules at chest CT is crucial for optimizing management in lung cancer screening. Purpose To develop and validate a deep learning...
- 1 August 2021
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 300 (2) , 438-447
Top Cited PapersDeep Learning for Malignancy Risk Estimation of Pulmonary Nodules Detected at Low-Dose Screening CT
A deep learning algorithm, developed for malignancy risk estimation of pulmonary nodules detected at screening CT, demonstrated good discriminative performance in a large independent validation set. Background Accurate estimation of the malignancy risk of pulmonary nodules at chest CT is crucial for optimizing management in lung cancer screening. Purpose To develop and validate a deep learning...
- 1 January 2021
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 298 (1) , E18-E28
Automated Assessment of COVID-19 Reporting and Data System and Chest CT Severity Scores in Patients Suspected of Having COVID-19 Using Artificial Intelligence
Background The COVID-19 pandemic has spread across the globe with alarming speed, morbidity and mortality. Immediate triage of suspected patients with chest infections caused by COVID-19 using ches...
- 1 January 2021
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 298 (1) , E18-E28
Automated Assessment of COVID-19 Reporting and Data System and Chest CT Severity Scores in Patients Suspected of Having COVID-19 Using Artificial Intelligence
Background The COVID-19 pandemic has spread across the globe with alarming speed, morbidity and mortality. Immediate triage of suspected patients with chest infections caused by COVID-19 using ches...