Su Hwan Kim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Su Hwan Kim has a total of 198 co-authors. Here are the top co-authors by number of shared publications:
- Benedikt Wiestler
Technical University of Munich
16 shared publications Claus Zimmer
Institute of Diagnostic and Interventional Neuroradiology, TUM University Hospital, School of Medicine and Health, Technical University of Munich, Munich, Germany
14 shared publicationsSeverin Schramm
Department of Diagnostic and Interventional Neuroradiology, TUM University Hospital, School of Medicine and Health, Technical University of Munich, Munich, Germany
14 shared publicationsDennis M. Hedderich
Department of Diagnostic and Interventional Neuroradiology, TUM University Hospital, School of Medicine and Health, Technical University of Munich, Munich, Germany
11 shared publicationsKeno K. Bressem
Institute for Diagnostic and Interventional Radiology, School of Medicine and Health, TUM Klinikum, Technical University of Munich (TUM), Munich, Germany
11 shared publicationsLisa C. Adams
Department of Diagnostic and Interventional Radiology, TUM University Hospital, School of Medicine and Health, Technical University of Munich, Munich, Germany
10 shared publications
Latest Publications
- 1 September 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 8 (5) , e260325
Human-AI Collaboration in Radiology: The Blind Spots
Despite rapid technological advances in radiology, radiologist–artificial intelligence interactions remain critically understudied. Three specific domains—cognitive psychology, user interface and user experience, and algorithmic conformity—demand dedicated empirical research.
- 1 September 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 8 (5) , e260325
Human-AI Collaboration in Radiology: The Blind Spots
Despite rapid technological advances in radiology, radiologist–artificial intelligence interactions remain critically understudied. Three specific domains—cognitive psychology, user interface and user experience, and algorithmic conformity—demand dedicated empirical research.
- Open Access
- 29 August 2026
- Published by Springer Nature in Communications Medicine
- Vol. 6 (1) , 467
Pooled two-cohort MRI body composition phenotyping with open-source deep learning
Background: Body mass index fails to capture variation in fat and muscle distribution that determines metabolic health and disease risk. MRI enables radiation-free quantification of regional body composition, yet scalable open-source tools applied in pooled cohorts with differing acquisition protocols have been lacking. Methods: MRSegmentator, an open-source nnU-Net-based pipeline, was applied to quantify visceral adipose tissue (VAT), abdominal subcutaneous adipose tissue (ASAT), gluteofemoral adipose tissue (GFAT), trunk musculature, and the liver mask used for liver fat-fraction estimation in 45,851 adults from the German National Cohort ( n = 26,877, 3 T multi-centre Siemens) and UK Biobank ( n = 18,974, 1.5 T Siemens). Population-scale compartment volumes were segmented from stitched in-phase gradient-echo (GRE) images in both cohorts; liver fat fraction was calculated from fat-only and water-only images. The annotated development data comprised NAKO T2-HASTE and UKB Dixon reconstructions. A single pooled model was applied without site-specific adaptation. A separate two-reader agreement study used 50 scans from these annotated development-sequence domains. Associations between BMI-adjusted body composition and cardiometabolic conditions were estimated using generalized linear mixed-effects models. Incremental discrimination beyond age, BMI, and waist-to-hip ratio was assessed. Results: Five-fold participant-stratified internal cross-validation against curated human-in-the-loop development references comprising UKB Dixon and NAKO T2-HASTE yielded a mean Dice of 0.91. In a separate 50-scan reader study on these annotated development-sequence images, overall reader–reader Dice was 0.937 and overall algorithm–reader Dice was 0.908. The trained pipeline was then used to segment compartment volumes from stitched in-phase GRE inputs in both cohorts, while liver fat fraction was calculated from fat-only and water-only images; direct sequence-matched validation on NAKO GRE was not performed. VAT showed the strongest positive associations with cardiometabolic conditions, while GFAT showed inverse associations, most prominently for type 2 diabetes (OR 0.69, 95% CI 0.66 to 0.72). Disease-specific body-composition phenotypes were identified, with type 2 diabetes characterized by elevated VAT, reduced GFAT, and increased liver fat. MRI-derived compartments modestly improved discrimination for type 2 diabetes and hyperlipidemia beyond anthropometric measures. Conclusions: A single open-source deep-learning pipeline enabled pooled body-composition phenotyping in two cohorts and captured distributional variation in fat and muscle beyond BMI. High agreement in internal cross-validation (mean Dice 0.91) and the separate two-reader study support the annotated development-sequence analysis, while the population-scale application identified distinct disease-associated phenotypes and modest incremental discrimination beyond conventional anthropometry.
- 29 August 2026
- Published by Springer Nature in Communications Medicine
- Vol. 6 (1) , 467
Open AccessPooled two-cohort MRI body composition phenotyping with open-source deep learning
Background: Body mass index fails to capture variation in fat and muscle distribution that determines metabolic health and disease risk. MRI enables radiation-free quantification of regional body composition, yet scalable open-source tools applied in pooled cohorts with differing acquisition protocols have been lacking. Methods: MRSegmentator, an open-source nnU-Net-based pipeline, was applied to quantify visceral adipose tissue (VAT), abdominal subcutaneous adipose tissue (ASAT), gluteofemoral adipose tissue (GFAT), trunk musculature, and the liver mask used for liver fat-fraction estimation in 45,851 adults from the German National Cohort ( n = 26,877, 3 T multi-centre Siemens) and UK Biobank ( n = 18,974, 1.5 T Siemens). Population-scale compartment volumes were segmented from stitched in-phase gradient-echo (GRE) images in both cohorts; liver fat fraction was calculated from fat-only and water-only images. The annotated development data comprised NAKO T2-HASTE and UKB Dixon reconstructions. A single pooled model was applied without site-specific adaptation. A separate two-reader agreement study used 50 scans from these annotated development-sequence domains. Associations between BMI-adjusted body composition and cardiometabolic conditions were estimated using generalized linear mixed-effects models. Incremental discrimination beyond age, BMI, and waist-to-hip ratio was assessed. Results: Five-fold participant-stratified internal cross-validation against curated human-in-the-loop development references comprising UKB Dixon and NAKO T2-HASTE yielded a mean Dice of 0.91. In a separate 50-scan reader study on these annotated development-sequence images, overall reader–reader Dice was 0.937 and overall algorithm–reader Dice was 0.908. The trained pipeline was then used to segment compartment volumes from stitched in-phase GRE inputs in both cohorts, while liver fat fraction was calculated from fat-only and water-only images; direct sequence-matched validation on NAKO GRE was not performed. VAT showed the strongest positive associations with cardiometabolic conditions, while GFAT showed inverse associations, most prominently for type 2 diabetes (OR 0.69, 95% CI 0.66 to 0.72). Disease-specific body-composition phenotypes were identified, with type 2 diabetes characterized by elevated VAT, reduced GFAT, and increased liver fat. MRI-derived compartments modestly improved discrimination for type 2 diabetes and hyperlipidemia beyond anthropometric measures. Conclusions: A single open-source deep-learning pipeline enabled pooled body-composition phenotyping in two cohorts and captured distributional variation in fat and muscle beyond BMI. High agreement in internal cross-validation (mean Dice 0.91) and the separate two-reader study support the annotated development-sequence analysis, while the population-scale application identified distinct disease-associated phenotypes and modest incremental discrimination beyond conventional anthropometry.
- Open Access
A 3-Minute Education on the False Positive Paradox Improves Trust Calibration in AI-Assisted Intracranial Aneurysm Detection: A Multinational Randomized Controlled Reader Study
Background: Even a highly accurate diagnostic test can yield more false-positive than true-positive findings in low-prevalence settings, which is known as the false positive paradox. Radiologists’ unawareness of this paradox may foster automation bias, the tendency to excessively rely on AI outputs. Methods: In this prospective, multinational, randomized controlled reader study (DRKS00038740), 34 readers from 10 countries (16 residents, 8 general radiologists or fellows, and 10 neuroradiologists) were randomly assigned to a control group (n = 17) or intervention group (n = 17), stratified by experience level. The intervention group reviewed a short, 3-minute educational video explaining the false positive paradox prior to the reading session. Both groups evaluated 20 TOF-MRA studies with AI-flagged findings (10% true-positive, 90% false-positive). Primary outcomes were acceptance rate of false-positive AI findings and follow-up intensity. These were evaluated using mixed models with crossed random effects for reader and case. Results: At baseline, readers vastly overestimated the positive predictive value of AI tools for intracranial aneurysm detection (mean estimate, 62.9%; simulation-based estimate, 15.4% [95% interval, 8.1-28.0%]). The intervention reduced the odds of accepting AI false positives (OR 0.50 [upper 95% confidence bound, 0.95], one-sided p = 0.017), with acceptance probabilities of 12.7% (95% CI, 6.0-25.0%) in the intervention group compared to 22.5% (95% CI, 11.6-39.2%) in the control group. The intervention group exhibited a downward shift in follow-up intensity for false positives (OR 0.47 [upper 95% confidence bound, 0.81]; one-sided p = 0.014), recommending follow-up in 39.2% (120/306) of cases, compared to 54.9% (168/306) in the control group. Conclusion: A brief education on the false positive paradox improved trust calibration in AI-assisted intracranial aneurysm detection. Our findings highlight the potential of reader-side cognitive debiasing strategies to improve trust calibration and support safer use of AI in radiology. Summary: A brief education on realistic positive predictive value ranges reduced radiologists’ uncritical acceptance of false-positive AI flags for intracranial aneurysm and the intensity of recommended follow-up. Key Results: - In a multinational, randomized controlled reader study with 34 radiologists from 10 countries, readers initially vastly overestimated the positive predictive value of AI for intracranial aneurysm detection (mean estimate, 62.9%; simulation-based estimate, 15.4% [95% interval, 8.1-28.0%]), demonstrating base-rate neglect. - Exposure to a 3-minute educational video explaining the false positive paradox reduced the odds of accepting false-positive AI findings (OR 0.50 [upper 95% confidence bound 0.95], one-sided p = 0.017), with an acceptance probability of 12.7% (95% CI, 6.0-25.0%) in the intervention group compared to 22.5% (95% CI, 11.6-39.2%) in the control group. - The intervention group had lower odds of recommending a more intensive follow-up strategy (OR 0.47 [upper 95% confidence bound, 0.81]; one-sided p = 0.014).
Open AccessA 3-Minute Education on the False Positive Paradox Improves Trust Calibration in AI-Assisted Intracranial Aneurysm Detection: A Multinational Randomized Controlled Reader Study
Background: Even a highly accurate diagnostic test can yield more false-positive than true-positive findings in low-prevalence settings, which is known as the false positive paradox. Radiologists’ unawareness of this paradox may foster automation bias, the tendency to excessively rely on AI outputs. Methods: In this prospective, multinational, randomized controlled reader study (DRKS00038740), 34 readers from 10 countries (16 residents, 8 general radiologists or fellows, and 10 neuroradiologists) were randomly assigned to a control group (n = 17) or intervention group (n = 17), stratified by experience level. The intervention group reviewed a short, 3-minute educational video explaining the false positive paradox prior to the reading session. Both groups evaluated 20 TOF-MRA studies with AI-flagged findings (10% true-positive, 90% false-positive). Primary outcomes were acceptance rate of false-positive AI findings and follow-up intensity. These were evaluated using mixed models with crossed random effects for reader and case. Results: At baseline, readers vastly overestimated the positive predictive value of AI tools for intracranial aneurysm detection (mean estimate, 62.9%; simulation-based estimate, 15.4% [95% interval, 8.1-28.0%]). The intervention reduced the odds of accepting AI false positives (OR 0.50 [upper 95% confidence bound, 0.95], one-sided p = 0.017), with acceptance probabilities of 12.7% (95% CI, 6.0-25.0%) in the intervention group compared to 22.5% (95% CI, 11.6-39.2%) in the control group. The intervention group exhibited a downward shift in follow-up intensity for false positives (OR 0.47 [upper 95% confidence bound, 0.81]; one-sided p = 0.014), recommending follow-up in 39.2% (120/306) of cases, compared to 54.9% (168/306) in the control group. Conclusion: A brief education on the false positive paradox improved trust calibration in AI-assisted intracranial aneurysm detection. Our findings highlight the potential of reader-side cognitive debiasing strategies to improve trust calibration and support safer use of AI in radiology. Summary: A brief education on realistic positive predictive value ranges reduced radiologists’ uncritical acceptance of false-positive AI flags for intracranial aneurysm and the intensity of recommended follow-up. Key Results: - In a multinational, randomized controlled reader study with 34 radiologists from 10 countries, readers initially vastly overestimated the positive predictive value of AI for intracranial aneurysm detection (mean estimate, 62.9%; simulation-based estimate, 15.4% [95% interval, 8.1-28.0%]), demonstrating base-rate neglect. - Exposure to a 3-minute educational video explaining the false positive paradox reduced the odds of accepting false-positive AI findings (OR 0.50 [upper 95% confidence bound 0.95], one-sided p = 0.017), with an acceptance probability of 12.7% (95% CI, 6.0-25.0%) in the intervention group compared to 22.5% (95% CI, 11.6-39.2%) in the control group. - The intervention group had lower odds of recommending a more intensive follow-up strategy (OR 0.47 [upper 95% confidence bound, 0.81]; one-sided p = 0.014).
- 1 July 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 8 (4) , e260529
Externally Tested AI for Lung Nodule Classification: A Realistic Benchmark for an Emerging Screening Era
- 1 July 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 8 (4) , e260529
Externally Tested AI for Lung Nodule Classification: A Realistic Benchmark for an Emerging Screening Era
- Open Access
Post Hoc Localization of Beam F3 Stimulation Targets: An MRI-Derived Geodesic Approach for Refined TMS E-Field Simulations
Background: Transcranial magnetic stimulation (TMS) targeting the left dorsolateral prefrontal cortex (dlPFC) is an established treatment option in major depressive disorder. One of the most common approaches for targeting the dlPFC is the Beam F3 method, which determines the stimulation site (F3 Beam ) as a function of external cranial measurements. Precise knowledge of the individual stimulation site is essential for imaging-based analyses of TMS effects. However, due to the method’s reliance on individual anatomy, retrospective identification of F3 Beam targets across cohorts is challenging, limiting the analysis of existing datasets. We developed a scalable method to reconstruct subject-specific F3 Beam target locations for e-field simulations based on structural imaging. Methods: High-resolution three-dimensional (3D) T1-weighted MRI was used to generate individual scalp meshes via the “Simulation of Non-Invasive Brain Stimulation” (SimNIBS) software. Subject-specific anatomical distances and coordinates of interest were measured geodesically using a Python-based script to reconstruct the individual F3 Beam targets. Validation included a retrospective comparison between digital geodesic measurements and manual cranial measurements in 20 patients and a prospective comparison with MR-visible scalp markers in 2 healthy controls. To assess the impact of our targeting algorithm on e-field simulations, volumetric e-field maps based on three potential targets (F3 Beam, F3 MNI , F3 Geo ) were generated in SimNIBS and compared using voxel-wise statistics in SPM12. Results: Retrospective analysis revealed a systematic bias towards higher in vivo measurements compared to digital geodesic measurements, though deviations in the final distances determining F3 Beam (x Beam and y Beam ) were minimal (Λx Beam : 0.11 ± 0.08 cm; Λy Beam : 0.14 ± 0.21 cm). Prospective validation demonstrated that F3 Beam coordinates better matched in vivo coil positions than group-template-derived targets (F3 MNI ). Group-level analysis showed method-dependent clustering of coil positions with corresponding voxel-wise e-field differences. Conclusions: Individualized geodesic measurements may enable accurate, scalable and retrospective identification of Beam F3 targets and coil orientations. This approach may yield more accurate e-field simulations than group-template based targeting and provides a practical method for retrospective analysis of existing TMS treatment cohorts. This could be leveraged to identify response predictors or imaging-based biomarkers of treatment response.
Open AccessPost Hoc Localization of Beam F3 Stimulation Targets: An MRI-Derived Geodesic Approach for Refined TMS E-Field Simulations
Background: Transcranial magnetic stimulation (TMS) targeting the left dorsolateral prefrontal cortex (dlPFC) is an established treatment option in major depressive disorder. One of the most common approaches for targeting the dlPFC is the Beam F3 method, which determines the stimulation site (F3 Beam ) as a function of external cranial measurements. Precise knowledge of the individual stimulation site is essential for imaging-based analyses of TMS effects. However, due to the method’s reliance on individual anatomy, retrospective identification of F3 Beam targets across cohorts is challenging, limiting the analysis of existing datasets. We developed a scalable method to reconstruct subject-specific F3 Beam target locations for e-field simulations based on structural imaging. Methods: High-resolution three-dimensional (3D) T1-weighted MRI was used to generate individual scalp meshes via the “Simulation of Non-Invasive Brain Stimulation” (SimNIBS) software. Subject-specific anatomical distances and coordinates of interest were measured geodesically using a Python-based script to reconstruct the individual F3 Beam targets. Validation included a retrospective comparison between digital geodesic measurements and manual cranial measurements in 20 patients and a prospective comparison with MR-visible scalp markers in 2 healthy controls. To assess the impact of our targeting algorithm on e-field simulations, volumetric e-field maps based on three potential targets (F3 Beam, F3 MNI , F3 Geo ) were generated in SimNIBS and compared using voxel-wise statistics in SPM12. Results: Retrospective analysis revealed a systematic bias towards higher in vivo measurements compared to digital geodesic measurements, though deviations in the final distances determining F3 Beam (x Beam and y Beam ) were minimal (Λx Beam : 0.11 ± 0.08 cm; Λy Beam : 0.14 ± 0.21 cm). Prospective validation demonstrated that F3 Beam coordinates better matched in vivo coil positions than group-template-derived targets (F3 MNI ). Group-level analysis showed method-dependent clustering of coil positions with corresponding voxel-wise e-field differences. Conclusions: Individualized geodesic measurements may enable accurate, scalable and retrospective identification of Beam F3 targets and coil orientations. This approach may yield more accurate e-field simulations than group-template based targeting and provides a practical method for retrospective analysis of existing TMS treatment cohorts. This could be leveraged to identify response predictors or imaging-based biomarkers of treatment response.
- 1 May 2026
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 319 (2) , e253477
Performing Best When Needed Least: Reader Experience Shapes Accuracy Gains in Large Language Model–assisted Brain MRI Differential Diagnosis
With increasing reader experience, large language model accuracy improved, whereas diagnostic gains from model assistance paradoxically diminished, highlighting the divergence between model performance and clinically relevant benefits for readers.
- 1 May 2026
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 319 (2) , e253477
Performing Best When Needed Least: Reader Experience Shapes Accuracy Gains in Large Language Model–assisted Brain MRI Differential Diagnosis
With increasing reader experience, large language model accuracy improved, whereas diagnostic gains from model assistance paradoxically diminished, highlighting the divergence between model performance and clinically relevant benefits for readers.
- 1 May 2026
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 319 (2) , e251347
Associations of MRI-derived Paraspinal IMAT and LMM with Cardiometabolic Risk Factors: Results from a German Cohort
Artificial intelligence–derived muscle composition from whole-body MRI scans identified individuals with cardiometabolic risk factors but no pre-existing conditions in a nationwide population.
- 1 May 2026
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 319 (2) , e251347
Associations of MRI-derived Paraspinal IMAT and LMM with Cardiometabolic Risk Factors: Results from a German Cohort
Artificial intelligence–derived muscle composition from whole-body MRI scans identified individuals with cardiometabolic risk factors but no pre-existing conditions in a nationwide population.
- Open Access
- 1 January 2026
- Published by BMJ in BMJ Digital Health & AI
- Vol. 2 (1) , e000189
Exposing the fragility of LLM reasoning through bias-inducing prompts: evidence from BiasMedQA
Objectives: To evaluate the impact of large language model (LLM) reasoning on model susceptibility to cognitive bias-inducing prompts. Methods and analysis: The performance of Llama-3.3-70B, Qwen3-32B and Gemini-2.5-Flash, along with their reasoning-enhanced variants, was evaluated in the public BiasMedQA dataset developed to evaluate seven established cognitive biases in 1273 clinical case vignettes. Each model was tested using a base prompt, a debiasing prompt with the instruction to actively mitigate cognitive bias and a few-shot prompt with additional sample cases of biased responses. Beyond the seven biases from BiasMedQA, Gemini-2.5-Flash was additionally tested using four unpublished bias-inducing prompts to unveil signs of potential data contamination and actively investigate brittleness. For each model pair, two mixed-effects logistic regression models were fitted to determine the impact of biases and mitigation strategies on performance. Results: In all three models, the reasoning-enhanced variant achieved higher rates of correct responses (Llama-3.3-70B: 72.5–82.1% vs 61.0–73.4%, Qwen3-32B: 71.7–78.7% vs 55.5–64.1%, Gemini-2.5-Flash: 81.8–88.6% vs 80.0–83.7%). The performance of Gemini-2.5-Flash dropped considerably when exposing it to four additional unpublished bias-inducing prompts (from 80.0–88.6% to 47.4–86.1%), hinting at potential contamination of its training data and exposing underlying brittleness. In Llama-3.3-70B and Gemini-2.5-Flash, reasoning amplified model vulnerability to several bias-inducing prompts, while reasoning reduced susceptibility of Qwen3-32B to one of the seven biases. The debiasing and few-shot prompting approaches demonstrated statistically significant reductions in biased responses across all three model architectures. Conclusion: In none of the three LLMs, reasoning was able to consistently reduce vulnerability to bias-inducing prompts, revealing the fragility of the reasoning capabilities purported by the model developers.
- 1 January 2026
- Published by BMJ in BMJ Digital Health & AI
- Vol. 2 (1) , e000189
Open AccessExposing the fragility of LLM reasoning through bias-inducing prompts: evidence from BiasMedQA
Objectives: To evaluate the impact of large language model (LLM) reasoning on model susceptibility to cognitive bias-inducing prompts. Methods and analysis: The performance of Llama-3.3-70B, Qwen3-32B and Gemini-2.5-Flash, along with their reasoning-enhanced variants, was evaluated in the public BiasMedQA dataset developed to evaluate seven established cognitive biases in 1273 clinical case vignettes. Each model was tested using a base prompt, a debiasing prompt with the instruction to actively mitigate cognitive bias and a few-shot prompt with additional sample cases of biased responses. Beyond the seven biases from BiasMedQA, Gemini-2.5-Flash was additionally tested using four unpublished bias-inducing prompts to unveil signs of potential data contamination and actively investigate brittleness. For each model pair, two mixed-effects logistic regression models were fitted to determine the impact of biases and mitigation strategies on performance. Results: In all three models, the reasoning-enhanced variant achieved higher rates of correct responses (Llama-3.3-70B: 72.5–82.1% vs 61.0–73.4%, Qwen3-32B: 71.7–78.7% vs 55.5–64.1%, Gemini-2.5-Flash: 81.8–88.6% vs 80.0–83.7%). The performance of Gemini-2.5-Flash dropped considerably when exposing it to four additional unpublished bias-inducing prompts (from 80.0–88.6% to 47.4–86.1%), hinting at potential contamination of its training data and exposing underlying brittleness. In Llama-3.3-70B and Gemini-2.5-Flash, reasoning amplified model vulnerability to several bias-inducing prompts, while reasoning reduced susceptibility of Qwen3-32B to one of the seven biases. The debiasing and few-shot prompting approaches demonstrated statistically significant reductions in biased responses across all three model architectures. Conclusion: In none of the three LLMs, reasoning was able to consistently reduce vulnerability to bias-inducing prompts, revealing the fragility of the reasoning capabilities purported by the model developers.
- 1 November 2025
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 317 (2) , e251844
A Prospective Controlled Trial of Large Language Model–based Simplification of Oncologic CT Reports for Patients with Cancer
Large language model–based simplification of oncologic CT reports substantially improved participant comprehension and reduced reading burden but required radiologist oversight because of clinically relevant errors in 3%–7% of simplified reports.
- 1 November 2025
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 317 (2) , e251844
A Prospective Controlled Trial of Large Language Model–based Simplification of Oncologic CT Reports for Patients with Cancer
Large language model–based simplification of oncologic CT reports substantially improved participant comprehension and reduced reading burden but required radiologist oversight because of clinically relevant errors in 3%–7% of simplified reports.
- Open Access
The Expertise Paradox: Who Benefits from LLM-Assisted Brain MRI Differential Diagnosis?
Purpose To evaluate how reader experience influences the diagnostic benefit from LLM assistance in brain MRI differential diagnosis. Materials and Methods Neuroradiologists (n = 4), radiology residents (n = 4), and neurology/neurosurgery residents (n = 4) were recruited. A dataset of complex brain MRI cases was curated from the local imaging database (n = 40). For each case, readers provided a textual description of the main imaging finding and their top three differential diagnoses ("Unassisted"). Three state-of-the-art large language models (GPT-4.1, Gemini 2.5 Pro, DeepSeek-R1) were prompted to generate top-three differentials based on the clinical case description and reader-specific findings. Readers then revised their differential diagnoses after reviewing GPT-4.1 suggestions ("Assisted"). To evaluate the association between reader experience and diagnostic benefit, a cumulative link mixed model (CLMM) was fitted, with change in diagnostic result as ordinal outcome, reader experience as predictor, and random intercepts for rater and case. Results LLM-generated differential diagnoses achieved the highest top-3 accuracy when provided with image descriptions from neuroradiologists (top-3: 78.8-83.8%), followed by radiology residents (top-3: 71.8-77.6%), and neurology/neurosurgery residents (top-3: 62.6-64.5%). In contrast, mean relative gains in top-3 accuracy through LLM assistance diminished with increasing experience, with +19.2% for neurology/neurosurgery residents (from 43.2% to 62.6%), +14.7% for radiology residents (from 59.6% to 74.4%), and +4.4% for neuroradiologists (from 83.1% to 87.5%). The CLMM demonstrated a significant negative association between reader experience and diagnostic benefit from LLM assistance (β = −0.10, p = 0.005). Conclusion With increasing reader experience, absolute diagnostic LLM performance with reader-generated input improved, while relative diagnostic gains through LLM assistance paradoxically diminished. Our findings call attention to the divergence between standalone LLM performance and clinically relevant reader benefit, and emphasize the need to account for human-AI interaction in this context.
Open AccessThe Expertise Paradox: Who Benefits from LLM-Assisted Brain MRI Differential Diagnosis?
Purpose To evaluate how reader experience influences the diagnostic benefit from LLM assistance in brain MRI differential diagnosis. Materials and Methods Neuroradiologists (n = 4), radiology residents (n = 4), and neurology/neurosurgery residents (n = 4) were recruited. A dataset of complex brain MRI cases was curated from the local imaging database (n = 40). For each case, readers provided a textual description of the main imaging finding and their top three differential diagnoses ("Unassisted"). Three state-of-the-art large language models (GPT-4.1, Gemini 2.5 Pro, DeepSeek-R1) were prompted to generate top-three differentials based on the clinical case description and reader-specific findings. Readers then revised their differential diagnoses after reviewing GPT-4.1 suggestions ("Assisted"). To evaluate the association between reader experience and diagnostic benefit, a cumulative link mixed model (CLMM) was fitted, with change in diagnostic result as ordinal outcome, reader experience as predictor, and random intercepts for rater and case. Results LLM-generated differential diagnoses achieved the highest top-3 accuracy when provided with image descriptions from neuroradiologists (top-3: 78.8-83.8%), followed by radiology residents (top-3: 71.8-77.6%), and neurology/neurosurgery residents (top-3: 62.6-64.5%). In contrast, mean relative gains in top-3 accuracy through LLM assistance diminished with increasing experience, with +19.2% for neurology/neurosurgery residents (from 43.2% to 62.6%), +14.7% for radiology residents (from 59.6% to 74.4%), and +4.4% for neuroradiologists (from 83.1% to 87.5%). The CLMM demonstrated a significant negative association between reader experience and diagnostic benefit from LLM assistance (β = −0.10, p = 0.005). Conclusion With increasing reader experience, absolute diagnostic LLM performance with reader-generated input improved, while relative diagnostic gains through LLM assistance paradoxically diminished. Our findings call attention to the divergence between standalone LLM performance and clinically relevant reader benefit, and emphasize the need to account for human-AI interaction in this context.