JKJiyeong Kim

Jiyeong Kim

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jiyeong Kim has a total of 136 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Secure Patient Messaging in Ophthalmology: Retrospective Natural Language Processing Study

    by Jiyeong Kim, Zoha Zahid Fazal, Jibreel Obaid Rehman, Sophia Y Wang, Robert Chang, Eleni Linos, Yasir Sepah

    Background: The rapid rise of secure patient messaging has transformed ophthalmic communication, with text volumes now exceeding prepandemic levels across health care systems. Our study uses a natural language processing (NLP) model to explore patterns in over 30,000 secure messages and identify common ocular complaints and differences by patient demographics. Objective: The objective of this study is to characterize the clinical and administrative concerns communicated through secure ophthalmology messaging and to assess differences in message content across patient sociodemographic groups. Methods: We conducted a cross-sectional study of deidentified, patient-initiated secure messages sent between June 2014 and July 2024. Patients with ophthalmic conditions who initiated secure electronic health record (EHR) portal messages were included. Of the 52,388 extracted message threads, 30,390 (58%) patient medical advice request (PMAR) messages routed solely to the Department of Ophthalmology from 4817 unique patients were assessed after the exclusion of questionnaires, courtesy messages, clinician responses, and messages routed to other health care departments. Of these unique patients, 2673 (55.5%) were female, 2740 (56.9%) were aged 50 years or older, 2346 (48.7%) were White, 4392 (91.2%) were non-Hispanic, and 2774 (57.6%) were married. NLP and large language model (LLM)–assisted topic classification were used to categorize message content. The distribution of message topics and frequency of clinical concerns were assessed and stratified by age, sex, race, ethnicity, and marital status. Differences in message frequency by demographic subgroup were assessed using 2-proportion z-tests. Results: Following preprocessing and topic modeling, 14,284 (47%) of the 30,390 messages could be predefined in relation to a topic cluster and were included in the final analysis. Nearly half of all 14,284 predefined messages addressed administrative issues, including 4166 (29.2%) scheduling-related inquiries, 1352 (9.5%) pharmacy and medication refill requests, and 581 (4.1%) insurance document issues. Among clinical concerns, of the total 14,284 predefined messages, 2975 (20.8%) messages involved vision disturbances, 1241 (8.7%) involved glaucoma-related symptoms, 1074 (7.5%) involved tumor-related questions, and 1064 (7.4%) involved surgery-relevant concerns. Message content differed significantly by demographic characteristics. Non-White patients more frequently raised issues related to pharmacy refills, insurance, glaucoma, and disability documentation, whereas White patients more often reported surgical concerns. Older patients more frequently messaged about glaucoma, surgery, and tumor-related issues, while female patients more often reported complications and swelling or infection. Conclusions: Secure patient messages frequently include clinically relevant symptoms with potential triage implications and demonstrate demographic differences in care-seeking behavior. Systematic analysis of message content may support safer triage, improved workflow efficiency, and more equitable delivery of ophthalmic care.


  • Open Access

    Health Care Professionals’ Perceptions of AI in Clinical Practice: Productivity, Enjoyment, and Pay

    by Michael L Chen, Jiyeong Kim, Mariana Ramirez Posada, David Entwistle, Isabella de Vere Hunt, Yifan Li, Xue Li, Nathanael J Fast, Fatima Rodriquez, Albert Chiou, et al.

    Despite widespread discussion of opportunities and risks about AI in medicine, few health care professionals routinely used AI during this study period. Physicians and men were more likely to report frequent AI use, and frequent AI users more often reported positive sentiments about AI’s future impacts on pay, enjoyment, and productivity at work. The youngest professionals (≤29 years) were less engaged and more skeptical about AI’s future benefits. These findings highlight a gap between AI’s promise and medical practice, as adoption varies by role, experience, and demographics.

  • Open Access

    Artificial Intelligence for Evidence Synthesis of Emerging Biologics to Improve Skeletal Health in Osteogenesis Imperfecta: Systematic Review and Meta-Analysis

    by Chengfei Li, Zonglin Dai, Wing Chung Tang, Zesen Gao, Vivien Kin Yi Chan, Mariana Ramirez-Posada, Jiyeong Kim, Eleni Linos, Cl Cheung, Ian Chi Kei Wong, et al.

    Background: Osteogenesis imperfecta (OI) is a rare genetic disorder characterized by bone fragility and recurrent fractures. Emerging biologics demonstrate promise by targeting bone-remodeling pathways, yet evidence for their efficacy and safety remains fragmented and heterogeneous, and no prior systematic review in OI has incorporated artificial intelligence (AI) to synthesize it. Objective: This study aims to systematically evaluate the efficacy and safety of novel biologics in patients with OI using an AI-assisted workflow for evidence synthesis. Methods: We conducted a systematic review and meta-analysis of interventional trials of denosumab, setrusumab, teriparatide, romosozumab, and fresolimumab. Data were retrieved from PubMed, Web of Science, Embase, ScienceDirect, the Cochrane Library, and ClinicalTrials.gov up to December 1, 2025. Eligible studies enrolled individuals with OI, reported areal bone mineral density (aBMD) and/or fractures, and were randomized, nonrandomized, or single-arm studies; case series were excluded. As a methodological feature, GPT-4o was integrated into the workflow to perform a parallel 2-stage screening (title/abstract and full text) and to assist with risk of bias assessment using an adapted Cochrane RoB 2 tool. The primary outcome, percentage change in aBMD, was synthesized using a random-effects meta-analysis. GPT-4o was benchmarked against human reviewers using sensitivity, specificity, and weighted Cohen κ. Results: Thirteen trials (n=684) were systematically reviewed, of which 10 (n=333) contributed to meta-analyses. In children, denosumab produced the greatest 12-month increase in lumbar spine aBMD (25.49%, 95% CI 17.14%‐33.84%). In adults, setrusumab at 12 months yielded the highest improvement (9.38%, 95% CI 6.5%‐12.26%). Across trials, no biologic significantly reduced fracture incidence compared to bisphosphonates. Safety profiles varied: denosumab was associated with a high risk of hypercalcemia in children (30.95%), whereas setrusumab had no treatment-related serious adverse events. AI achieved high sensitivity in abstract (97.4%) and full-text (88.9%) screening, and reduced total screening time by over 95%. Although there was substantial agreement with humans in the quality assessment (Cohen κ=0.778, 95% CI 0.710‐0.846), the model exhibited optimism and positional biases due to reliance on probabilistic language patterns rather than structured clinical reasoning. Conclusions: This review is the first to synthesize and quantitatively compare skeletal outcomes across multiple biologics in OI with an AI-assisted review workflow. Denosumab and setrusumab demonstrate promising efficacy in improving lumbar spine aBMD across ages, although current evidence does not support superior fracture reduction over bisphosphonates. GPT-4o can substantially accelerate evidence synthesis but should be deployed with explicit human oversight in tasks requiring contextual understanding and clinical reasoning. These findings should be interpreted cautiously given the small and heterogeneous trial base. Taken together, our workflow presented how evidence synthesis may be scaled and operationalized in real-world rare disease research.

  • Towards AI-augmented decision making in psychiatry

    by Jiyeong Kim, Carolyn I. Rodriguez

    Psychiatric disorders are heterogeneous, and care depends on interpreting unstructured longitudinal narratives, creating variability that hinders standardization. A study now shows that a psychiatry-specific large language model (LLM) may help clinicians to deliver more consistent, high-quality care.

  • Open Access

    Unseen Insights: An AI-Powered Exploration of Secure Patient Messages in Ophthalmology (Preprint)

    by Jiyeong Y Kim, Zoha Z Fazal, Sophia Y Wang, Robert T Chang, Eleni Linos, Yasir J Sepah

    BACKGROUND Patient portal messaging has rapidly expanded as a key mode of communication in modern health systems, generating large volumes of unstructured patient-generated health data. However, the clinical relevance and operational implications of these messages in ophthalmology remain poorly characterized. OBJECTIVE To characterize the clinical and administrative concerns communicated through secure ophthalmology messaging and to assess differences in message content across patient sociodemographic groups. METHODS This is a cross-sectional study of de-identified, patient-initiated secure messages sent between June 2014 and July 2024. Patients with ophthalmic conditions who initiated secure electronic health record portal messages. Of 48,516 extracted message threads, 30,390 patient medical advice request messages from 4,817 unique patients were included after exclusion of questionnaires, courtesy messages, and clinician responses. Natural language processing and large language model–assisted topic classification were used to categorize message content. Differences in message frequency by demographic subgroup were assessed using 2-proportion z tests. RESULTS Participants were 55.5% female, 56.9% aged 50 years or older, 48.7% White, and 85.7% non-Hispanic. Nearly half of all messages addressed administrative issues, including scheduling, medication refills, and insurance. Among clinical concerns, vision disturbances (20.8%), glaucoma-related symptoms (8.7%), imaging or tumor-related questions (7.5%), and postoperative concerns (7.4%) were most common. Message content differed significantly by demographic characteristics. Non-White patients more frequently raised issues related to pharmacy refills, insurance, glaucoma, and disability documentation, whereas White patients more often reported surgical concerns. Older patients more frequently messaged about glaucoma, surgery, and tumor-related issues, while female patients more often reported complications and swelling or infection. CONCLUSIONS Secure patient messages frequently include clinically relevant symptoms with potential triage implications and demonstrate demographic differences in care-seeking behavior. Systematic analysis of message content may support safer triage, improved workflow efficiency, and more equitable delivery of ophthalmic care.

  • Open Access

    Evaluating Diagnostic Accuracy and Clinical Reasoning of Multiple Large Language Models in Psychiatry

    by Kevin W. Jin, Yasna Rostam-Abadi, Pooja Chaudhary, Margaret A. Garrett, Ashley S. Huang, Mario Montelongo, Caesa Nagpal, Jasperina Shei, Judah Weathers, Juliana S. Zhang, et al.

    Summary: Background: Existing large language model (LLM) evaluations rely on accuracy benchmarks that fail to capture whether models reason well while making diagnoses. Studies that do analyse reasoning focus on post hoc explanations accompanying model outputs rather than distinct, clinician-visible artifacts such as detailed reasoning traces. This creates a translational gap in domains such as psychiatry, where diagnosis relies on narrative interpretation, diagnostic reasoning, and clinical judgment under uncertainty. Methods: We conducted a mixed-methods evaluation of four state-of-the-art LLMs using a clinician-curated dataset of 196 psychiatric case vignettes, including 135 published cases and 61 novel clinician-authored vignettes. Diagnostic accuracy was assessed using multiple metrics (top-1 accuracy, top-5 accuracy, recall@5, and mean reciprocal rank) based on ranked lists of five differential diagnoses per vignette. Clinical reasoning quality was evaluated on a randomly selected subset of 30 vignettes through clinician assessment of model-generated diagnostic reasoning traces alongside qualitative commentary from board-certified psychiatrists. We examined the association between clinician-rated reasoning quality and diagnostic correctness and included an illustrative comparison with psychiatry residents. Findings: Clinician-rated diagnostic reasoning quality was strongly associated with diagnostic correctness in mixed-effects logistic regression analyses (β = 1·80; p < 0·001), whereas data extraction quality alone was not. Across the full vignette set, models demonstrated moderate to high diagnostic accuracy. The highest-performing model achieved a top-5 accuracy of 0·801 and also received the highest clinician-rated reasoning scores. In an illustrative comparison, model diagnostic accuracy fell within the range observed for psychiatry residents. Interpretation: Diagnostic reasoning quality captures clinically meaningful variation in LLM performance beyond accuracy metrics. Psychiatry may represent a stringent testbed for evaluating reasoning in narrative-driven clinical domains. Evaluation frameworks for LLM-based clinical decision support should incorporate structured assessment of reasoning processes, not accuracy alone. Research in Context: Evidence before this study: We searched PubMed and Scopus for studies evaluating large language models for psychiatric diagnosis and/or differential diagnosis from text vignettes. Searches were run from database inception to February 6, 2026, using terms including (“large language model” OR LLM “artificial intelligence” OR “generative AI” OR AI OR ChatGPT OR GPT OR Claude OR Gemini OR DeepSeek OR Llama) AND (psychiatr* OR mental OR DSM OR “differential diagnosis” OR diagnos*) AND (vignette OR case OR “case report”). We included empirical studies that evaluated model diagnostic outputs using psychiatric cases/vignettes and excluded editorials/commentaries and studies that did not report case-level diagnostic performance. We did not formally assess risk of bias/quality; studies were heterogeneous in vignette sources, model access, and outcome definitions, precluding quantitative pooling. Prior studies have typically examined small vignette sets, focused on narrow diagnostic domains, evaluated single models, or relied primarily on outcome-based accuracy metrics. When diagnostic reasoning has been assessed, it has usually been inferred from post hoc explanations accompanying model outputs rather than evaluated as a distinct, clinician-visible artifact. Clinician-grounded evaluations of diagnostic reasoning across multiple contemporary models remain limited. Added value of this study: This study provides a large-scale, clinician-grounded evaluation of diagnostic accuracy and diagnostic reasoning quality across four contemporary large language models using a diverse dataset of psychiatric case vignettes. Rather than relying solely on outcome-based explanations, we directly evaluated model-generated diagnostic reasoning traces as clinician-visible artifacts using structured clinician ratings and qualitative analysis. By integrating multiple accuracy metrics with clinician assessment of reasoning coherence, flexibility, and plausibility, we demonstrate that clinician-rated reasoning quality is strongly associated with diagnostic correctness, whereas data extraction quality alone is not. Our analysis also identifies recurrent reasoning failure modes not captured by accuracy metrics, highlighting psychiatry as a stringent testbed for evaluating reasoning in narrative-driven clinical domains. Implications of all the available evidence: Evaluations of large language models for clinical decision support should extend beyond accuracy to include systematic assessment of clinician-visible diagnostic reasoning. Mixed-methods, clinician-grounded evaluation frameworks that examine both diagnostic outcomes and reasoning artifacts may be critical for responsible assessment of LLMs in psychiatry and other areas of medicine where diagnosis depends on interpretation, judgment, and tolerance of uncertainty.


  • Open Access

    Optimizing large language models for detecting symptoms of depression/anxiety in chronic diseases patient communications

    by Jiyeong Kim, Stephen P. Ma, Michael L. Chen, Isaac R. Galatzer-Levy, John Torous, Peter J. van Roessel, Christopher Sharp, Michael A. Pfeffer, Carolyn I. Rodriguez, Eleni Linos, et al.

    Patients with diabetes are at increased risk of comorbid depression or anxiety, complicating their management. This study evaluated the performance of large language models (LLMs) in detecting these symptoms from secure patient messages. We applied multiple approaches, including engineered prompts, systemic persona, temperature adjustments, and zero-shot and few-shot learning, to identify the best-performing model and enhance performance. Three out of five LLMs demonstrated excellent performance (over 90% of F-1 and accuracy), with Llama 3.1 405B achieving 93% in both F-1 and accuracy using a zero-shot approach. While LLMs showed promise in binary classification and handling complex metrics like Patient Health Questionnaire-4, inconsistencies in challenging cases warrant further real-life assessment. The findings highlight the potential of LLMs to assist in timely screening and referrals, providing valuable empirical knowledge for real-world triage systems that could improve mental health care for patients with chronic diseases.


  • Open Access

    Leveraging Large Language Models and Patient Portal Messages for Early Identification of Depression

    by Jiyeong Kim, John Torous, Julia Adler-Milstein, Peter J. van Roessel, Fatima Rodriguez, Christopher Sharp, Michael A. Pfeffer, Jonathan H. Chen, Stephen P. Ma, Carolyn I. Rodriguez, et al.

    Importance Large language model (LLM)-assisted early warning system may help overcome existing barriers to timely depression diagnosis in patients with cardiovascular disease (CVD). This novel application of LLMs to screen patient messages could be applied to other chronic diseases, facilitating automated symptom-driven diagnoses and interventions. Objective To prospectively simulate the impact (change in time to diagnosis) of population mental health screening using LLMs by screening patient portal messages, and measure LLMs’ accuracy to identify individuals at high risk for depression diagnosis in patients with CVD. Design Prospective cohort study Setting Electronic health records from an academic hospital (Stanford Health Care) Participants Individuals with CVD diagnosed 2014-2024, subsequently diagnosed with depression Intervention/Exposure LLMs (Llama 3.1 8B, July 2024 version, Meta LLC, and MedGemma 4B, July 2025 version, Google DeepMind, LLC) to identify individuals with depression Main outcome Accuracy of LLMs in sensitivity for completeness to capture positive cases and positive predictive value (PPV) for correctness to capture positive cases, and changes in time to depression diagnosis Results We identified 115,156 patients with CVD, and 23.1% (N = 26,578/115,156) of those had co-morbid depression. We included individuals (n=2,314) who sent at least one message between CVD and depression diagnoses. Participants were mostly 65 years and older (n = 1,718/2,314, 74.2%), or of non-Hispanic ethnicity (N = 2,078/2,314, 89.9%), or of the White race (N = 1,506/2,314, 66.1%), but sex was balanced (females, N = 1,197/2,314, 51.7%). PPV was 51.2% [95% CI: 47.5-54.5%] Llama 3.1 8B, and sensitivity was 83.6% [81.1-85.9] Llama 3.1 8B and 71.0% [67.0-75.3] MedGemma 4B. On average, the LLM (Llama 3.1 8B) detected depression 660 days earlier than the first charted diagnosis over a 1,746-day assessment period, a typical timeline from CVD to depression diagnoses in our cohort. Conclusion/Relevance LLMs identified individuals with depression significantly earlier than official diagnosis among patients with CVD, relying solely on longitudinal patient messages without additional medical information, with high sensitivity and Patient Health Questionnaire-9 comparable PPV. This novel approach is applicable to various diagnoses. Questions Among those with cardiovascular disease, can large language models (LLMs) identify individuals with depression earlier than the first charted diagnosis via patient messages, and how much sooner and accurately? How frequently are depression symptoms detected before diagnosis? Findings LLMs identified individuals with depression 660 days earlier than the official diagnosis, with promising sensitivity (83%) and Patient Health Questionnaire-9 comparable PPV (51%) Meaning LLMs applied to patient messages showed strong potential for early diagnosis as a screening assistant, which is a novel use for this data source that could be expanded to other diagnoses.

  • Open Access

    Assessing the performance of artificial intelligence models in evaluating inflammatory skin disease severity: a systematic review and meta-analysis

    by Zhuo Ran Cai, Jiyeong Kim, Shawheen J Rezaei, Michael L Chen, Fadi Touma, Catherine Zhu, Sonia Onyeka, Fonette Fonjungo, Jesutofunmi A Omiye, Isabelle Krakowski, et al.

    Background: Artificial intelligence (AI) applications in dermatology have expanded beyond diagnosis and have shifted towards assessing disease severity. Objectives: To qualitatively and quantitatively evaluate the performance of image-based AI models in severity assessment for various skin diseases. Methods: In this systematic review and meta-analysis, we collected studies using four electronic databases, including PubMed, Embase, Institute of Electrical and Electronics Engineers Xplore and Web of Science, published from 1 January 2017 to 6 April 2023, and updated the search in November 2023. Studies assessing the performance of deep learning AI models on the severity of skin diseases were included. We excluded studies that utilized a nonvalidated severity index, lacked clinical images, and assessed wounds, ulcers or burns. Two independent reviewers extracted prespecified study characteristics for the summary table. For the meta-analysis, contingency tables were extracted, when possible, and reconstructed for each severity measure. Accuracy was calculated using a bivariate model in Metandi package, and meta-regression was performed by disease type and scoring system. This study was registered with PROSPERO (CRD42023487228). Results: Our initial search identified 7737 records. After duplicate removal and abstract screening, we reviewed the full text of 192 articles and included 45 studies for systematic review and 19 for meta-analysis. The pooled sensitivity and specificity of AI models were 80.5% [95% confidence interval (CI) 76.2–84.2] and 96.2% (95% CI 94.9–97.2), respectively. Moreover, pooled sensitivity differed by disease (atopic dermatitis 91.8% vs. acne 80.7%, P = 0.005; acne 80.4% vs. psoriasis 71.1%, P = 0.044) and scoring system [Eczema Area and Severity Index 97.3% vs. Investigator’s Global Assessment (IGA) for atopic dermatitis 78.9%, P < 0.001; Hayashi Grading 89.7% vs. IGA for acne 69.8%, P < 0.001]. Conclusions: Our findings show that current AI models exhibit a high level of capacity in disease severity assessment. Nevertheless, efforts are urgently needed to improve transparency in data reporting and conduct high-quality prospective studies using objective reference standards in clinical settings to generate reliable evidence.


  • Open Access

    Artificial Intelligence in Obsessive-Compulsive Disorder: A Systematic Review

    by Jiyeong Kim, Juan Pablo Gonzalez Pacheco, Ashleigh Golden, Elias Aboujaoude, Peter van Roessel, Aayushi Gandhi, Pavithra Mukunda, Tatevik Avanesyan, Haopeng Xue, Ehsan Adeli, et al.

    Purpose of Review: Obsessive-compulsive disorder (OCD) is a chronic and disabling condition, often leading to significant functional impairments. Despite its early onset, there is an average delay of 17 years from symptom onset to diagnosis and treatment, resulting in poorer outcomes. This systematic review aims to synthesize current findings on the application of AI in OCD, highlighting opportunities for early symptom detection, scalable therapy training, clinical decision support, novel therapeutics, computer vision-based approaches, and multimodal biomarker discovery. Recent Findings: While previous reviews focused on biomarker-based OCD detection and treatment using machine learning (ML), the findings of the current review add information about novel applications of deep learning technology, specifically generative artificial intelligence (GenAI) and natural language processing (NLP). Among the included 13 articles, most studies (84.6%) utilized secondary data analyses, primarily through GenAI/NLP. Nearly 77% of these studies were published in the past two years, with high quality of evidence. The primary focus areas were enhancing treatment and management, and timely OCD detection (both 38.5%); followed by AI tool development for broader mental health applications. Summary: AI technologies offer transformative potential for improvements related to OCD if diagnosis occurs earlier after onset; thereby lessening the consequential economic burden. Prioritizing investment in ethically sound AI research could significantly improve OCD outcomes in mental health care.