Danielle Lee Mowery

Despite the critical role of mental health services in suicide prevention, disparities in service utilization persist across various individual and social determinants of health. This study identifies key factors and intersectional patterns associated with mental healthcare use among young adults with past-year suicidal ideation, by developing machine learning-based artificial intelligence to improve predictive accuracy while ensuring interpretability for clinicians and policymakers. Utilizing cross-sectional data from the 2015–2020 National Survey on Drug Use and Health, we analyzed 11,018 US young adults aged 18–34 who reported past-year suicidal ideation. Random Forest and Shapley Additive Explanations identified the strongest predictors of mental healthcare utilization among 23 individual and social determinants of health. The decision tree model visualized prediction of utilization rates across intersectional characteristics. Findings indicate that depression, race/ethnicity, sexual orientation, and private health insurance were the strongest predictors of service use. Individuals without depression, males, Black, Indigenous, and People of Color, heterosexual individuals, and individuals without private health insurance were significantly less likely to seek care. These findings highlight the current intersectional disparities in mental healthcare utilization among young adults at risk of suicide. Expanding culturally competent care and promoting equitable access to mental healthcare for young adults at risk are crucial steps in addressing these disparities.

BACKGROUND Social determinants of health (SDOH) profoundly influence cancer and cardiovascular disease (CVD) risk, yet systematic extraction of SDOH from unstructured electronic health records (EHR) remains challenging. Traditional natural language processing (NLP) approaches require extensive manual annotation and institution-specific training data, while active SDOH screening imposes burden on patients and clinicians and is prone to missingness. Large language models (LLM) offer a promising alternative, but existing LLM-based SDOH extraction studies lack patient-level validation, are limited to constrained note types, and have not been validated against patient-reported outcomes. OBJECTIVE To develop and validate an LLM-based NLP framework for extracting SDOH from EHR at both the individual note and patient level, with comprehensive error analysis and comparison across both clinician-documented and patient-reported measures. METHODS From three independent cohorts, 100 patients were sampled and categorized into five distinct batches, and 97 patients yielded meaningful notes to construct the final dataset. Batch one was used for prompt development to evaluate five LLMs, with the best model then validated on the remaining four batches. Five SDOH domains were extracted: employment, financial toxicity, distress, food insecurity, and physical activity. Validation errors were categorized by a trained clinician at both the note-level (n=284) and patient-level (n=56). Agreement between clinician-documented and patient-reported SDOH measures was also assessed. RESULTS GPT-4.1 achieved 0.94 accuracy in the development batch. Across validation batches 2–5, GPT-4.1 achieved a note-level accuracy of 0.94 (SD 0.01; mean across batches) across the five SDOH domains, with an overall κ=0.72. Mean patient-level accuracy across validation batches was 0.88 (SD 0.04) for the five domains. Clinician-validated error analysis at both the note and patient level identified incorrect inferences as the predominant error type (note level, 57.7%; patient level, 44.6%). Clinician-documented and patient-reported SDOH showed moderate agreement (κ=0.17–0.52), with under-documentation of financial toxicity (58.8% agreement) and food insecurity (78.4%), and over-documentation of distress (69.1%). CONCLUSIONS GPT-4.1 provides validated SDOH extraction at both the note and patient level. Most errors stemmed from model interpretation rather than the model’s inability to find relevant information. Moderate agreement between documentation sources (patient-reported vs. clinician-documented) highlights their complementary value and the need to improve clinician documentation of SDOH domains. CLINICALTRIAL NCT01173341, NCT05078190, NCT05096338

We built and evaluated a zero-shot LLM pipeline with automated, task-aware prompt optimization to extract radiology and symptom fields for gallstone phenotyping from de-identified EHR text. Across symptomatic, asymptomatic, and control cohorts, it performed reliably on high-signal binary fields and symptom flags but lagged on fine-grained stone burden and complications, establishing a practical baseline and motivating targeted refinements

Background: The use of artificial intelligence (AI) to analyze health care data has become common in behavioral health sciences. However, the lack of training opportunities for mental health professionals limits clinicians’ ability to adopt AI in clinical settings. AI education is essential for trainees, equipping them with the literacy needed to implement AI tools in practice, collaborate effectively with data scientists, and develop skills as interdisciplinary researchers with computing skills. Objective: As part of the Penn Innovation in Suicide Prevention Implementation Research Center, we developed, implemented, and evaluated a virtual workshop to educate psychiatry and psychology trainees on using AI for suicide prevention research. Methods: The workshop introduced trainees to natural language processing (NLP) concepts and Python coding skills using Jupyter notebooks within a secure Microsoft Azure Databricks cloud computing and analytics environment. We designed a 3-hour workshop that covered 4 key NLP topics: data characterization, data standardization, concept extraction, and statistical analysis. To demonstrate real-world applications, we processed chief complaints from electronic health records to compare the prevalence of suicide-related encounters across populations by race, ethnicity, and age. Training materials were developed based on standard NLP techniques and domain-specific tasks, such as preprocessing psychiatry-related acronyms. Two researchers drafted and demonstrated the code, incorporating feedback from the Methods Core of the Innovation in Suicide Prevention Implementation Research to refine the materials. To evaluate the effectiveness of the workshop, we used the Kirkpatrick program evaluation model, focusing on participants’ reactions (level 1) and learning outcomes (level 2). Confidence changes in knowledge and skills before and after the workshop were assessed using paired t tests, and open-ended questions were included to gather feedback for future improvements. Results: A total of 10 trainees participated in the workshop virtually, including residents, postdoctoral researchers, and graduate students from the psychiatry and psychology departments. The participants found the workshop helpful (mean 3.17 on a scale of 1‐4, SD 0.41). Their overall confidence in NLP knowledge significantly increased ( P =.002) from 1.35 (SD 0.47) to 2.79 (SD 0.46). Confidence in coding abilities also improved significantly ( P =.01), increasing from 1.33 (SD 0.60) to 2.25 (SD 0.42). Open-ended feedback suggested incorporating thematic analysis and exploring additional datasets for future workshops. Conclusions: This study illustrates the effectiveness of a tailored data science workshop for trainees in psychiatry and psychology, focusing on applying NLP techniques for suicide prevention research. The workshop significantly enhanced participants’ confidence in conducting data science research. Future workshops will cover additional topics of interest, such as working with large language models, thematic analysis, diverse datasets, and multifaceted outcomes. This includes examining how participants’ learning impacts their practice and research, as well as assessing knowledge and skills beyond self-reported confidence through methods such as case studies for deeper insights.

Background: The safety planning intervention (SPI) is a suicide prevention intervention that results in a written plan to help patients reduce suicide risk. High-quality safety plans—that is, those that are the most complete, personalized, and specific—are more effective in reducing suicide risk. Measuring SPI quality is labor-intensive, which means that clinicians rarely get specific, actionable feedback on their use of the SPI. Objective: This study aimed to develop the Safety Plan Fidelity Rater, an automated tool that assesses the quality of written safety plans leveraging 3 large language models (LLMs)—GPT-4, LLaMA 3, and o3-mini. Methods: Using 266 deidentified safety plans from outpatient mental health settings in New York, LLMs analyzed four key steps: warning signs, internal coping strategies, making the environment safe, and reasons for living. We compared the predictive performance of the three LLMs, optimizing scoring systems, prompts, and parameters. Results: Findings showed that LLaMA 3 and o3-mini outperformed GPT-4, with different step-specific scoring systems recommended based on weighted F ₁ -scores. Conclusions: These findings highlight LLMs’ potential to provide clinicians with timely and accurate feedback on safety plan quality, which could greatly improve its implementation in community practice.

BACKGROUND Despite bearing the largest burden of suicide globally, low- and middle-income countries (LMICs) remain significantly underrepresented in suicide research compared to high-income countries (HICs). OBJECTIVE This study leverages natural language processing (NLP) and qualitative analysis to examine disparities in suicide research between LMICs and HICs with a particular focus on suicidal ideation (SI). Additionally, the study aims to identify themes in LMIC SI research and explore trends within these themes over time. METHODS An analysis of 15,708 articles published between 1968 and 2022 was conducted, extracting country of focus using article titles and abstracts through NLP methods. Among the 1,458 articles focusing on SI in LMICs, Latent Dirichlet Allocation (LDA) and manual qualitative theming were used to identify and examine thematic clusters. RESULTS Results show that only 26.6% of SI articles focused on LMICs while 70.5% focused on HICs. Among the papers focusing on LMICS, five key themes emerged, including: developmental and lifespan context; interpersonal and risk factors; clinical instruments and care; gender, sex, perinatal and discrimination; and suicide attempts and death. Analyses were also used to identify trends in how often these themes were discussed within LMIC research over time. CONCLUSIONS Findings highlight the need for increased SI research in LMICs in additional to improved research infrastructure and support. This study also demonstrates the utility of NLP and qualitative methodologies for large-scale research syntheses and provides promising directions for future global disparities research.

In-hospital cardiac arrest (IHCA) affects over 200,000 patients annually in the United States, yet its detection through manual chart review remains resource-intensive and often delayed. We evaluated the performance of four open-source large language models (LLMs) and GPT-4o in identifying IHCA cases from 2,674 clinical notes across five hospitals. While GPT-4o achieved the highest performance (F1-score: 0.90, recall: 0.97), several open-source models demonstrated comparable capabilities, suggesting their viability for clinical applications. Our systematic analysis of model outputs revealed that performance was strongly influenced by site-specific documentation practices, with inter-site agreement rates varying by over 20%. Through detailed error analysis, we identified key challenges including medical terminology hallucinations and structural inconsistencies in model reasoning. These findings establish a framework for implementing LLM-based IHCA detection systems while highlighting critical considerations for their clinical deployment.

Background: Automated bidirectional SMS text messaging has emerged as a compelling strategy to facilitate communication between patients and the health system after hospital discharge. Understanding the unique ways in which patients interact with these messaging programs can inform future efforts to tailor their design to individual patient styles and needs. Objective: Our primary aim was to identify and characterize distinct patient interaction phenotypes with a postdischarge automated SMS text messaging program. Methods: This was a secondary analysis of data from a randomized controlled trial that tested a 30-day postdischarge automated SMS text messaging intervention. We analyzed SMS text messages and patterns of engagement among patients who received the intervention and responded to messages. We engineered features to describe patients’ engagement with and conformity to the program and used a k-means clustering approach to learn distinct interaction phenotypes among program participant subgroups. We also looked at the association between these interaction phenotypes and (1) patient demographics and clinical characteristics and (2) hospital revisit outcomes. Results: A total of 1731 patients engaged with the intervention, among which 1060 (61.2%) were female; the mean age was 65 (SD 16.1) years; 782 (45.2%) and 828 (47.8%) patients identified as Black and White, respectively; and 970 (56%) and 317 (18.3%) patients were insured by Medicare and Medicaid, respectively. Using k-means clustering, we observed four distinct subgroups representing patient interaction phenotypes: (1) a high engagement, high conformity group (enthusiasts, n=1029); (2) a low engagement, high conformity group (minimalists, n=515); (3) a low engagement, low conformity group (nonadapters, n=170); and (4) a high engagement with an intense level of need group (high needs responders, n=17). Differences were observed in demographic characteristics—including gender, race, and insurance type—and clinical outcomes across groups. Conclusions: For health systems looking to leverage an SMS text messaging approach to engage patients after discharge, this work offers two main takeaways: (1) not all patients interact with SMS text messaging equally, and some may require either additional guidance or a different medium altogether; and (2) the way in which patients interact with this type of program (in addition to the information they communicate through the program) may have added predictive signal toward adverse outcomes.