Christopher Kitchen

Objective: A diversity of approaches and cohorts have been published in research literature for suicide risk estimation, leading to some uncertainty around which factors and conditions are most strongly contributing to risk. Methods exist for weighting estimates by propensity score, removing simplifying the interpretation of model effects. Few studies have explored the use of these tools to assess when the effect of certain factors is uncertain or inconsistent. Method: A mix of 1,001,737 suicide decedents and living patients were evaluated for suicide risk, relying on a subset of commonly identified factors: demographics, psychiatric conditions, substance use disorders, and social needs. Care utilization patterns were then assessed controlling for these factors in a hospital care setting. Four methods were considered for controlling multivariate confounding of odds predicting death by suicide: multivariate logistic regression, 1-to-1 sample matching, inverse propensity weighting (IPTW), and entropy balancing (EBAL). Results: Test statistics showed suicide decedents were much more likely to have various psychiatric and substance use diagnoses, social needs and more likely to heightened care utilization across sampling intervals. Greater utilization was found close to date of death. Effect sizes after IPTW and EBAL weighting were also generally greater than would have been documented by regression alone. Conclusions: Propensity weighting is a useful alternative to exploring nuanced effects, especially when the direction and strength of an association differ from traditional statistics. By weighting, the overall imbalance of cases remains consistent and effect terms might reflect more nuanced associations than regression or sample matching alone.

Background: The use of artificial intelligence and machine learning (ML) tools is now common in the advancement of health care services and clinical risk estimation. Legacy systems make use of highly informative feature sets developed from years of clinical expertise and research to estimate different outcomes, but only recently have they been tested against novel statistical approaches. One such system, the Johns Hopkins Adjusted Clinical Group (ACG) System, is a long-standing and widely used approach to categorizing clinical risk factors, and it is amenable to ML techniques. Objective: This study aims to test the ACG System using a contrasted area under the receiver operating characteristic (AUROC) and F ₁ classification optimization strategy and compare its performance against traditional logistic regression methods. Assuming that selected ML algorithms can be tuned to enhance overall measures of performance, this would strengthen arguments for incorporating them into ACG-related workflows. Methods: Using a retrospective observational design, prospective year estimates of all-cause hospitalization and elevated total cost were modeled using a cross-validation framework. Patients with elevated costs were identified as those falling above the 95th percentile of total amounts billed, including pharmacy costs. Hyperparameter settings for XGBoost (Extreme Gradient Boosting), random forest, and elastic net were determined using average cross-validated performances for F ₁ and AUROC in a grid search aimed at maximizing either statistic. Additional iterated cross-validation was used to compare point-estimated average AUROC and F ₁ -scores between models, further decomposed by sensitivity, positive predictive value, and F -beta statistics. Results: There were 350,463 patients selected in 2019 from the Johns Hopkins Health System. Model features identified by the ACG System for predicting prospective year hospitalization and total cost were included in these analyses. Findings suggest small but statistically significant improvements in cross-validated AUROC and F ₁ -scores over logistic regression, using either optimization strategy and XGBoost. Logistic models achieved average receiver operating characteristic values of 0.886 and 0.841 for cost and hospitalization, respectively, whereas XGBoost achieved 0.891 and 0.849, respectively. F ₁ optimization yielded similar findings, with logistic models achieving 0.367 and 0.341 on average for hospitalization and cost, respectively, but XGBoost exceeded values for cost but not for hospitalization (0.411 and 0.328, respectively). Conclusions: The clinical implications of these findings and the effect of class imbalance on model calibration are explored, along with the limitations of these data and approach. The core finding is that logistic regression remains very well-suited to these tasks, especially in situations where the efficiency or interpretability of models is critical. Under conditions of imbalance, regressions tended to yield high-precision estimates for the outnumbered class. Nevertheless, the findings also underscore a diversity of suitable models depending on clinical use cases, each having its own tradeoffs for evaluating performance. As such, health systems must clearly identify the needs and expectations of a model before calibrating one for use.

The online version contains supplementary material available at 10.1038/s41598-026-50095-z.

Objectives: Research in suicide risk prediction often suffers from the lack of comprehensive data on patient suicide death, which differs from suicide attempt or suicidal behaviors. This study aimed to develop a population-wide multi-source harmonized data warehouse suitable for suicide death risk prediction. Materials and Methods: The Maryland Suicide Data Warehouse (MSDW) was conceived as a statewide database that addresses limitations in prior suicide research. To develop MSDW, multiple patient-level statewide data sources were linked using the statewide health information exchange infrastructure. Manner of death, the standard outcome in suicide death research, was determined by the state medical examiner. Health services data were linked from multiple data sources such as electronic health records, hospital discharge data, and administrative insurance claims. Data were structured as a common format that preserves observations at their lowest level of analysis. Data features were included based on known or hypothesized psychiatric or suicide risk factors. Results: The warehouse contains a mix of records across data sources for patient diagnoses, clinical encounters, procedures, area of residence, pharmacy fills and laboratory findings. MSDW represents 104,517 decedents reported by the OCME between 2012 and 2020, 5,059 classified as suicides. Conclusions: The MSDW is a statewide data warehouse that allows users to conduct population health research, predictive modeling and observational studies for multiple outcomes. It has multiple overlapping clinical records that improve the completeness and timeliness of data. It is a high-quality statewide data warehouse for conducting suicide prediction research and assessing risk for surveillance and intervention.

Background: Accurate classification of intentional death as suicide is essential to target prevention measures appropriately. Unfortunately, manner of death (MOD) for many opioid-related fatalities are unclassified in the United States, and in Maryland, as many as 82% of overdose deaths are classified as undetermined manner. Aims: For opioid-related fatalities in Maryland, leverage death certificate data to develop a model for identifying suicide as MOD among those classified as undetermined by the medical examiner. Method: Demographic and toxicology data were used to develop a classification model for opioid-related deaths where MOD was known, and then applied to a cohort where MOD was undetermined to estimate the likelihood that the intent was suicide. Results: Antidepressants, neuroleptics, oxycodone, benzodiazepines, and acetaminophen were more common in deaths classified as suicide while fentanyl, cocaine, and morphine were more common among accidental deaths. Our classification model correctly identified suicide cases 82% of the time (PPV = 0.82; AUC = 0.90) and expanded the number of suicide cases by 43% when applied to undetermined deaths. Limitations: The accuracy and completeness of death records. Conclusions: Data from standard autopsies can be used to detect additional suicide deaths with good statistical precision. Incorporating clinical information could enhance predictive accuracy and improve classification.

Background: The implementation of suicide risk models is challenging because the conditions in which they are developed often do not reflect those in which they are being used. The setting of an arbitrary classification threshold limits the interpretability of predictions, and their associated performance statistics. This work endeavors to explore different class imbalance ratios, across training sample compositions, time horizons and patient characteristics to understand how degree of imbalance affects the associated performance of regression-based predictive models of suicide. Patients and setting: The study population included 1,649,577 patients who were selected from the Maryland Suicide Data Warehouse (MSDW) between 2016 and 2020. The MSDW contains clinical and demographic features derived from claims (Maryland Health Care Commission, MHCC)and hospital discharge records Health Services Cost Review Commission (HSCRC), for decedents and living patients within the state of Maryland. Suicide death was our primary outcome of interest in a cross validated framework stratified by sources of data in the MSDW. Results: Cross validated AUROC was not found to vary consistently with respect to training sample imbalance nor time horizon, but both were found to have a direct association with AUPRC. Indeed, AUPRC increased with greater sample imbalance for training or outcome horizon (AUPRC 0.246; 0.246; 0.593 for all decedents, HSCRC, and MHCC respectively). Stratified samples revealed no significant cross validated performance than the overall sample for AUROC (0.832; 0.913; 0.927, for decedents, HSCRC and MHCC). However, AUPRC was significantly greater when limiting our HSCRC and MHCC samples to patients seen in the emergency room (AUPRC 0.417; 0.782) or in the inpatient settings (0.371; 0.773), or patients who had ICD-10-CM coded social needs (0.479, HSCRC only). Performance was significantly worse when restricting samples to patients aged less than 18 years (AUPRC 0.036; 0.208, HSCRC and MHCC respectively). Conclusion: A low precision for estimated suicide risk can be understood as a consequence of some tradeoffs during model development, particularly training models with matched cases, balanced classes or within short time horizons. This work demonstrates the improved AUPRC performance of regression models in a cross validated framework when these conditions are made more realistic, in the context of class imbalance or less restrictive in that of time horizon. Additionally, we illustrate using the same data that training models in certain clinical cohorts (e.g., defined by age, care utilization and social need) can lead to robustly different estimates for precision and recall, but not AUROC.

A key vulnerability in modeling suicide death is a lack of precision and therefore estimates are thought as ultimately unhelpful to clinicians, even with more advanced or nuanced machine learning (ML) techniques. We sought to fill several conceptual gaps by assessing performance, focusing on the precision-recall tradeoff, across multiple techniques, and with ad hoc contextualization for sensitivity, cost-balance, and fairness. To identify robust, differential performances of a cross section of ML techniques on a suicide risk task, emphasizing overall AUPRC maximization and downstream effects on hypothetical decision support. A retrospective cohort was selected for patients receiving care or having died per the Office of the Medical Examiner (OCME), between 2017 and 2020 using the Maryland Suicide Datawarehouse (MSDW). AUPRC-optimized settings yielded cross-validated AUPRC significantly improved over logistic regressions, especially for XGBoost in both hospital discharge (AUPRC: 0.667; PPV: 0.941) and commercial claims records (AUPRC: 0.558; PPV: 0.857). F-Beta statistics revealed that when precision is preferred (e.g., 99.9 percentile), XGBoost are among the most efficient tools, while random forest and MLP are better when sensitivity is preferred (90 percentile or lower). No algorithmic bias was identified by age, sex or race, but significant changes in performance are noted with certain clinical characteristics. To our knowledge, this is the first use of an AUPRC-maxima optimization for ML tools with predicting suicide death. The utility of suicide risk models in clinical decision support is discussed as being tied to innate class imbalance challenges in model training, with recommendations being provided on how to better evaluate performance.

Background: Although the use of telehealth has declined since the pandemic, it remains a popular mode of care delivery across the cancer care continuum. Understanding telehealth in the context of cancer care is essential, as its benefits and challenges may differ among diverse population groups and geographic areas. Objective: This study aimed to examine patterns of telehealth utilization across the cancer care continuum and to identify factors associated with the receipt of telehealth in a large patient population. This study also aimed to assess the telehealth's impact on key health care delivery outcomes. Methods: We used an annualized retrospective cohort design using patient data from the Johns Hopkins Health System (JHHS), a large regional academic health center in Maryland. The study analyzed electronic health record (EHR) data covering the period from January 1, 2019, to December 31, 2023. Chronic conditions were defined through the Johns Hopkins Adjusted Clinical Groups (ACG) System, which identifies comorbidities based on the International Classification of Diseases, Tenth Revision, Clinical Modification, codes in the electronic health record. In addition, we used publicly available geospatial data (eg, internet connectivity, rural–urban commuting area) to assess telehealth receipt associations. Statistical modeling, including generalized estimating equations, was used to evaluate variations in telehealth utilization and outcomes. Results: A total of 124,974 adult patients receiving cancer-related care at Johns Hopkins Health System were identified during the study period. Telehealth users were significantly older (52.2% aged ≥65 years, 19,942 patients) compared to nonusers (48.7%, 42,209 patients). In addition, these users were more likely to be male (45.4%, 17,365 patients vs 40.2%, 34,839 patients) and to identify as White (70.8%, 27,071 patients vs 64.7%, 56,122 patients). Telehealth users also had a higher prevalence of comorbidities, with 61.5% (23,503 patients) reporting 3 or more chronic conditions compared to 38.0% (33,000 patients) among nonusers. A positive correlation was noted between rural–urban commuting area codes and telehealth service utilization (ρ=0.36; P P P <0.05, respectively). Adjusted analyses indicated that concurrent telehealth use was associated with reduced odds of emergency department visits (0.916, 95% CI 0.884-0.948) and hospitalizations (0.830, 95% CI 0.799-0.863), acknowledging the potential influence of residual confounding. Conclusions: Telehealth has emerged as a crucial mode of care delivery for patients with complex conditions such as cancer. Understanding usage patterns and factors influencing telehealth across the cancer care continuum, including geographic barriers, is vital to optimizing its implementation and ensuring health care systems meet the diverse needs of patients with cancer in a value-based care environment.

Objectives: To evaluate and compare different dimensionality reduction techniques for quantifying housing conditions as a social determinant of health (SDOH) across various geographic levels in the United States. Materials and Methods: A total of 15 housing characteristics from the American Community Survey data were analyzed at county, ZIP code, and Census tract levels. The robustness of 3 dimensionality reduction techniques was assessed in reducing the 15 housing characteristics into 1 housing score. These summarization methods included principal component analysis (PCA), t-distributed stochastic neighbor embedding (tSNE), and uniform manifold approximation and projection (UMAP). We visualized geographic distributions of the housing scores, assessed methodological discrepancies between the techniques, and analyzed agreement between housing characteristic variability and housing score variability. Results: The selected dimensionality reduction methods generated housing scores that demonstrated acceptable face validity when visualized through choropleth maps. The PCA method provided the most stable and consistent results across geographic levels. The PCA method also resulted in the highest correlation between the variability of the underlying housing characteristics and the summarized housing score. Discussion: Data-driven summarization techniques provide an alternative approach to traditional expert-based indices in capturing housing conditions as a single SDOH factor. In this study, among the different summarized housing scores, the PCA-generated score offered superior robustness, persistent data structure, and higher stability across years. Conclusion: Principal component analysis was identified as the most reliable and interpretable approach for summarizing housing conditions across geographic levels. These findings contribute to the methodological foundation required to develop robust SDOH measures that can inform public health policies and address health disparities.