CSCheong Shin

Cheong Shin

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Cheong Shin has a total of 4 co-authors. Here are the top co-authors by number of shared publications:

  • Hong Seon Lee

    Department of Radiology, Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul, Republic of Korea

    2 shared publications
  • Jung Hyun Park

    Department of Rehabilitation Medicine, Gangnam Severance Hospital, Rehabilitation Institute of Neuromuscular Disease, Yonsei University College of Medicine, Seoul, Korea

    2 shared publications
  • Sungjun Kim

    Department of Radiology, Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul, Republic of Korea

    2 shared publications
  • Young Han Lee

    Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science (CCIDS), Yonsei University College of Medicine, Seoul, Korea

    2 shared publications

Latest Publications

  • Radiologist Detection of Deficiencies in LLM-Generated Patient Communications About Radiology Reports: A Multi-Reader Study.

    by Cheong Shin, Jung Hyun Park, Sungjun Kim, Young Han Lee, Hong-Seon Lee

    Patients increasingly use large language models (LLMs) to interpret radiology reports, yet the reliability of radiologist oversight in detecting errors in patient-facing LLM responses remains unquantified. This study evaluated the diagnostic accuracy of radiologists in detecting deficiencies in LLM-generated answers to simulated patient questions and quantified the incremental benefit of multi-reader oversight. In this retrospective multi-reader study, three board-certified radiologists independently evaluated 1200 LLM-generated answers (from three different models; 400 per model) to questions derived from 187 MIMIC-IV radiology reports. Answers were scored on a 3-point scale: 1 (clinically significant error), 2 (incomplete/ambiguous), and 3 (accurate). An adjudicated consensus served as the reference standard. Individual reader sensitivity, specificity, and multi-reader miss rates were compared. Of 1200 answers, 113 (9.4%) were deficient by consensus, including 18 clinically significant errors (1.5%) and 95 incomplete/ambiguous responses (7.9%). For clinically significant errors, readers missed 5.6-11.1% of cases. Across all deficiencies, individual reader sensitivity ranged from 81.4% (95% CI, 73.3-87.5) to 96.5% (95% CI, 91.3-98.6), with significant differences (McNemar p < 0.001). A single reader missed an average of 12.4% of deficiencies, whereas dual-reader review reduced the average miss rate to 0.6%; triple-reader review detected all deficiencies. Individual radiologist review of LLM-generated patient communications showed variable sensitivity, with some clinically significant errors remaining undetected. Dual-reader oversight significantly improves the safety and reliability of LLM-based communication tools.

  • Open Access

    Patient-Facing Radiology Communication with LLMs: Calibration Deficit and the Metadata Paradox

    by Cheong Shin, Jung Hyun Park, Sungjun Kim, Young Han Lee, Hong-Seon Lee

    Background/Objectives: Patients increasingly access radiology reports via online portals and frequently seek clarification. While Large Language Models (LLMs) may facilitate this communication, their clinical safety and reliability in this context remain largely uncharacterized. This study aimed to evaluate performance heterogeneity (the disparity between factual synthesis and interpretive reasoning), the Metadata Paradox (performance degradation triggered by demographic priors), and calibration characteristics in answering simulated patient questions derived from radiology reports. Methods: In this retrospective study, 2000 simulated inquiries were generated from 200 MIMIC-IV radiology reports based on an expert-refined 10-category taxonomy, categorized into factual tasks (e.g., terminology/anatomy) and interpretive tasks (e.g., diagnostic confidence/finding detail). Three LLMs (GPT-4o mini, Grok (v4-0709), Claude 3.5 Sonnet) generated 12,000 answers (with/without metadata). Quality was scored (1–3 scale) by Gemini 2.5 Flash, validated by three independent board-certified radiologists and finalized through four-specialist consensus adjudication (n = 1200). Performance and self-confidence calibration were assessed using Generalized Estimating Equations. Results: The LLM judge showed an overall agreement rate of 90.5% with the adjudicated ground truth. Grok and Claude 3.5 Sonnet significantly outperformed GPT-4o mini (p < 0.001); specifically, GPT-4o mini was associated with a 2.8-fold higher risk of failure compared to Grok (adjusted OR 2.83; 95% CI: 2.28–3.49; p < 0.001) and an absolute risk difference (ARD) of 8.4 percentage points. Accuracy reached its ceiling in factual tasks (Terminology: 98.1%) but was significantly lower in interpretive tasks (Diagnostic Confidence: 82.3%, p < 0.001). Metadata inclusion triggered the ‘Metadata Paradox,’ significantly increasing the risk of failure (OR 1.11; p = 0.044). A substantial calibration deficit (defined as the disconnect between self-confidence and accuracy) was observed; notably, the majority of safety-critical errors (Score 1: clinically significant misinformation; n = 131) were assigned high self-confidence (≥8/10; GPT-4o mini: 93.8%, Grok: 100%, Claude 3.5 Sonnet: 61.5%). Conclusions: Although LLMs accurately address factual queries, their consistent calibration deficit in safety-critical errors and susceptibility to stochastic stereotyping highlight the necessity of independent verification frameworks.