Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Di Sun has a total of 75 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Quantifying the impact of a computer‐aided diagnostic score on the clinical diagnosis of functional seizures

    by Wesley T. Kerr, Katherine N. McFarlane, Corinne H. Allas, Samuel W. Terman, Markus Reuber, Sung Hyun Seo, Payge Barnard, Adriana Y. Koek, Amir H. Karimi, Siddhika S. Sreenivasan, et al.

    Objective: The diagnosis of functional/dissociative seizures (FDS) without ictal video‐electroencephalography is challenging. The Functional/Dissociative Seizures Likelihood Score (FSLS) is a machine learning‐based diagnostic score that aims to help clinicians identify FDS. We evaluated whether a human‐in‐the‐loop implementation of the FSLS improved the performance of clinicians identifying FDS as compared to epileptic seizures (ES). Methods: We constructed 117 anonymized cases about patients with ictal video‐electroencephalography‐documented FDS, epilepsy, co‐occurring ES and FDS, or physiological seizurelike events. Text‐based clinical history was presented followed by the FSLS. Readers were asked the most likely diagnosis after each piece of information. We used mixture modeling combined with mixed effects logistic regression to perform data‐driven grouping of participants based on observed patterns of diagnostic performance. Results: Overall, 163 readers saw 1142 cases (median = 4 cases/reader), and 146 (90%) had a performance higher than chance. More formal training in seizures was associated with better performance (epileptologist accuracy = 67%, mental health clinician accuracy = 52%). Data‐driven groups including 66% of readers benefitted from the FSLS (accuracy improvement = 12%–15%, p < .05), including those in the reference and near highest baseline performance group. Other groups had no net change in performance ( p > .75). Significance: Clinicians with more formal seizure training identified possible FDS more accurately than others, but formal training did not guarantee high diagnostic performance. Two performance‐based groups, which included 66% readers, benefitted from the FSLS because they identified when to change their mind on the basis of the FSLS's suggestion. The implementation of machine learning in the diagnosis of FDS should focus on identifying clinical settings where it can effectively enhance clinicians' decision‐making.

  • Open Access

    Evaluating the reliability of large language models for clinical data extraction in bladder cancer prognosis

    by Di Sun, Lubomir Hadjiiski, Grace Bruno, John Gormley, Heang-Ping Chan, Elaine M. Caoili, Richard H. Cohan, Ajjai Alva, Rada Mihalcea, Chuan Zhou, et al.

    Advances in natural language processing (NLP) and machine learning could assist human users in clinical data extraction from unstructured electronic medical records (EMRs). This study investigates the accuracy and consistency of several Large Language Models (LLMs) - including Dolly, Vicuna, Llama, and GPT-4 - in extracting critical clinical information pertinent to bladder cancer survival prediction. Using EMRs from 163 bladder cancer patients, we assessed the impact on LLM performance by factors such as differences in the trained models, model evolution, input text length, and sequencing of case inputs. GPT-4 demonstrated superior performance with Fleiss’ Kappa values exceeding 0.97, accuracy consistently above 93%, and survival prediction metrics closely aligned with ground truth (AUC ± 0.02). Among offline models, Llama-2.0-13b and Llama-3.3-70b exhibited the highest reliability in both information extraction and survival prediction. This study underscores the potential of LLMs to automate clinical data extraction for predictive modeling while highlighting the challenges related to LLM variability and reliability.


  • Open Access

    Outcome Prediction Using Multi-Modal Information: Integrating Large Language Model-Extracted Clinical Information and Image Analysis

    by Di Sun, Lubomir Hadjiiski, John Gormley, Heang-Ping Chan, Elaine Caoili, Richard Cohan, Ajjai Alva, Grace Bruno, Rada Mihalcea, Chuan Zhou, et al.

    Survival prediction post-cystectomy is essential for the follow-up care of bladder cancer patients. This study aimed to evaluate artificial intelligence (AI)-large language models (LLMs) for extracting clinical information and improving image analysis, with an initial application involving predicting five-year survival rates of patients after radical cystectomy for bladder cancer. Data were retrospectively collected from medical records and CT urograms (CTUs) of bladder cancer patients between 2001 and 2020. Of 781 patients, 163 underwent chemotherapy, had pre- and post-chemotherapy CTUs, underwent radical cystectomy, and had an available post-surgery five-year survival follow-up. Five AI-LLMs (Dolly-v2, Vicuna-13b, Llama-2.0-13b, GPT-3.5, and GPT-4.0) were used to extract clinical descriptors from each patient’s medical records. As a reference standard, clinical descriptors were also extracted manually. Radiomics and deep learning descriptors were extracted from CTU images. The developed multi-modal predictive model, CRD, was based on the clinical (C), radiomics (R), and deep learning (D) descriptors. The LLM retrieval accuracy was assessed. The performances of the survival predictive models were evaluated using AUC and Kaplan–Meier analysis. For the 163 patients (mean age 64 ± 9 years; M:F 131:32), the LLMs achieved extraction accuracies of 74%~87% (Dolly), 76%~83% (Vicuna), 82%~93% (Llama), 85%~91% (GPT-3.5), and 94%~97% (GPT-4.0). For a test dataset of 64 patients, the CRD model achieved AUCs of 0.89 ± 0.04 (manually extracted information), 0.87 ± 0.05 (Dolly), 0.83 ± 0.06~0.84 ± 0.05 (Vicuna), 0.81 ± 0.06~0.86 ± 0.05 (Llama), 0.85 ± 0.05~0.88 ± 0.05 (GPT-3.5), and 0.87 ± 0.05~0.88 ± 0.05 (GPT-4.0). This study demonstrates the use of LLM model-extracted clinical information, in conjunction with imaging analysis, to improve the prediction of clinical outcomes, with bladder cancer as an initial example.


  • Open Access

    Survival Prediction of Patients with Bladder Cancer after Cystectomy Based on Clinical, Radiomics, and Deep-Learning Descriptors

    by Di Sun, Lubomir Hadjiiski, John Gormley, Heang-Ping Chan, Elaine M. Caoili, Richard H. Cohan, Ajjai Alva, Vikas Gulani, Chuan Zhou

    Accurate survival prediction for bladder cancer patients who have undergone radical cystectomy can improve their treatment management. However, the existing predictive models do not take advantage of both clinical and radiological imaging data. This study aimed to fill this gap by developing an approach that leverages the strengths of clinical (C), radiomics (R), and deep-learning (D) descriptors to improve survival prediction. The dataset comprised 163 patients, including clinical, histopathological information, and CT urography scans. The data were divided by patient into training, validation, and test sets. We analyzed the clinical data by a nomogram and the image data by radiomics and deep-learning models. The descriptors were input into a BPNN model for survival prediction. The AUCs on the test set were (C): 0.82 ± 0.06, (R): 0.73 ± 0.07, (D): 0.71 ± 0.07, (CR): 0.86 ± 0.05, (CD): 0.86 ± 0.05, and (CRD): 0.87 ± 0.05. The predictions based on D and CRD descriptors showed a significant difference (p = 0.007). For Kaplan–Meier survival analysis, the deceased and alive groups were stratified successfully by C (p < 0.001) and CRD (p < 0.001), with CRD predicting the alive group more accurately. The results highlight the potential of combining C, R, and D descriptors to accurately predict the survival of bladder cancer patients after cystectomy.


  • Open Access

    Computerized Decision Support for Bladder Cancer Treatment Response Assessment in CT Urography: Effect on Diagnostic Accuracy in Multi-Institution Multi-Specialty Study

    by Di Sun, Lubomir Hadjiiski, Ajjai Alva, Yousef Zakharia, Monika Joshi, Heang-Ping Chan, Rohan Garje, Lauren Pomerantz, Dean Elhag, Richard H. Cohan, et al.

    This observer study investigates the effect of computerized artificial intelligence (AI)-based decision support system (CDSS-T) on physicians’ diagnostic accuracy in assessing bladder cancer treatment response. The performance of 17 observers was evaluated when assessing bladder cancer treatment response without and with CDSS-T using pre- and post-chemotherapy CTU scans in 123 patients having 157 pre- and post-treatment cancer pairs. The impact of cancer case difficulty, observers’ clinical experience, institution affiliation, specialty, and the assessment times on the observers’ diagnostic performance with and without using CDSS-T were analyzed. It was found that the average performance of the 17 observers was significantly improved (p = 0.002) when aided by the CDSS-T. The cancer case difficulty, institution affiliation, specialty, and the assessment times influenced the observers’ performance without CDSS-T. The AI-based decision support system has the potential to improve the diagnostic accuracy in assessing bladder cancer treatment response and result in more consistent performance among all physicians.