KZKelly L. Zuromski

Kelly L. Zuromski

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Kelly L. Zuromski has a total of 168 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Smartphone Screen Time Characteristics in People With Suicidal Thoughts: Retrospective Observational Data Analysis Study

    by Marta Karas, Debbie Huang, Zachary Clement, Alexander J Millner, Evan M Kleiman, Kate H Bentley, Kelly L Zuromski, Rebecca G Fortgang, Dylan DeMarco, Adam Haim, et al.

    Background: Smartphone-based monitoring in natural settings provides opportunities to monitor mental health behaviors, including suicidal thoughts and behaviors. To date, most suicidal thoughts and behaviors research using smartphones has primarily relied on collecting so-called “active” data, requiring participants to engage by completing surveys. Data collected passively from smartphone sensors and logs may offer an objectively measured representation of an individual’s behavior, including smartphone screen time. Objective: This study aims to present methods for identifying screen-on bouts and deriving screen time characteristics from passively collected smartphone state logs and to estimate daily smartphone screen time in people with suicidal thinking, providing a more reliable alternative to traditional self-report. Methods: Participants (N=126; median age 22, IQR 16-33 years) installed the Beiwe app (Harvard University) on their smartphones, which passively collected phone state logs for up to 6 months after discharge from an inpatient psychiatric unit (adolescents) or emergency department visit (adults). We derived daily screen time measures from these logs, including screen-on time, screen-on bout duration, screen-off bout duration, and screen-on bout count. We estimated the mean of these measures across age subgroups (adults and adolescents), phone operating systems (Android and iOS), and monitoring stages after the discharge (first 4 weeks vs subsequent weeks). We evaluated the sensitivity of daily screen time measures to changes in the parameters of the screen-on bout identification method. Additionally, we estimated the impact of a daylight time change on minute-level screen time using function-on-scalar generalized linear mixed-effects regression. Results: The median monitoring period was 169 (IQR 42‐169) days. For adolescents and adults, mean daily screen-on time was 254.6 (95% CI 231.4-277.7) and 271.0 (95% CI 252.2-289.8) minutes, mean daily screen-on bout duration was 4.233 (95% CI 3.565-4.902) and 4.998 (95% CI 4.455-5.541) minutes, mean daily screen-off bout duration was 25.90 (95% CI 20.09-31.71) and 26.90 (95% CI 22.18-31.66) minutes, and mean daily screen-on bout count (natural logarithm transformed) was 4.192 (95% CI 4.041-4.343) and 4.090 (95% CI 3.968-4.213), respectively; there were no significant differences between smartphone operating systems (all P values were >.05). The daily measures were not significantly different for the first 4 weeks compared to the fifth week onward (all P values were >.05), except average screen-on bout in adults ( P value = .018). Our sensitivity analysis indicated that in the screen-on bout identification method, the cap on an individual screen-on bout duration has a substantial effect on the resulting daily screen time measures. We observed time windows with a statistically significant effect of daylight time change on screen-on time (based on 95% joint confidence intervals bands), plausibly attributable to sleep time adjustments related to clock changes. Conclusions: Passively collected phone logs offer an alternative to self-report measures for studying smartphone screen time characteristics in people with suicidal thinking. Our work demonstrates the feasibility of this approach, opening doors for further research on the associations between daily screen time, mental health, and other factors.


  • Detecting suicide risk among U.S. servicemembers and veterans: a deep learning approach using social media data

    by Kelly L. Zuromski, Daniel M. Low, Noah C. Jones, Richard Kuzma, Daniel Kessler, Liutong Zhou, Erik K. Kastman, Jonathan Epstein, Carlos Madden, Satrajit S. Ghosh, et al.

    Background: Military Servicemembers and Veterans are at elevated risk for suicide, but rarely self-identify to their leaders or clinicians regarding their experience of suicidal thoughts. We developed an algorithm to identify posts containing suicide-related content on a military-specific social media platform.Methods: Publicly-shared social media posts (n = 8449) from a military-specific social media platform were reviewed and labeled by our team for the presence/absence of suicidal thoughts and behaviors and used to train several machine learning models to identify such posts.Results: The best performing model was a deep learning (RoBERTa) model that incorporated post text and metadata and detected the presence of suicidal posts with relatively high sensitivity (0.85), specificity (0.96), precision (0.64), F1 score (0.73), and an area under the precision-recall curve of 0.84. Compared to non-suicidal posts, suicidal posts were more likely to contain explicit mentions of suicide, descriptions of risk factors (e.g. depression, PTSD) and help-seeking, and first-person singular pronouns.Conclusions: Our results demonstrate the feasibility and potential promise of using social media posts to identify at-risk Servicemembers and Veterans. Future work will use this approach to deliver targeted interventions to social media users at risk for suicide.


  • Perspectives of suicide loss survivors: Qualitative analysis of data from a psychological autopsy study of U.S. Army soldiers

    by Kelly L. Zuromski, Chelsey R. Wilks, Maha Al‐Suwaidi, Ellen Wittler, Lilly Scherban, Blake Hite, LaTashia Raymond, Catherine L. Dempsey, Murray B. Stein, Robert J. Ursano, et al.

    Introduction: Suicide loss survivors can provide information not otherwise available about the circumstances preceding a suicide. In this study, we analyzed interview data from suicide loss survivors collected as part of a psychological autopsy study of U.S. Army soldiers.Methods: Next‐of‐kin (NOK) (n = 61) and Army supervisors (SUP) (n = 107) of suicide decedents (n = 135) who had died in the last 2–3 months answered open‐ended questions about suicide risk factors, ideas for improving suicide prevention, and the impact of the suicide. Responses were coded using conventional content analysis methods to identify common themes.Results: Many NOK (30%) and SUP (50%) did not observe any signs of risk preceding the soldier's suicide. The most common idea regarding suicide prevention from SUP was that the suicide was inevitable, whereas NOK were more likely to emphasize the importance of increasing mental health treatment and reducing stigma. Both NOK and SUP reported negative effects of the suicide, but SUP reported some positive effects (e.g., increased unit connectedness).Conclusions: Results underscore the challenges of using informants to identify soldiers at high risk of suicide, given many respondents did not observe any warning signs. Findings also highlight attitudinal barriers present in the military that, if targeted, may increase soldiers' help‐seeking and willingness to disclose their risk.


  • Open Access

    Detecting suicide risk among U.S. Servicemembers and Veterans: A deep learning approach using social media data

    by Kelly L Zuromski, Daniel Mark Low, Noah Jones, Richard Kuzma, Daniel T. Kessler, Liutong Zhou, Erik K Kastman, Jonathan Epstein, Carlos Madden, Satrajit S Ghosh, et al.

    Background: Military Servicemembers and Veterans are at elevated risk for suicide, but rarely self-identify to their leaders or clinicians regarding their experience of suicidal thoughts. We developed an algorithm to identify suicidal Servicemembers and Veterans on a military-specific social media platform. Methods: Publicly-shared social media posts (n = 8449) from a military-specific social media platform were reviewed and labeled by our team for the presence/absence of suicidal thoughts and behaviors and used to train several machine learning models to identify such posts. Results: The best performing model was a deep learning (RoBERTa) model that incorporated post text and metadata and detected the presence of suicidal posts with relatively high sensitivity (.85), specificity (.96), precision (.64), F1 score (.73), and an area under the precision-recall curve of .84. Compared to non-suicidal posts, suicidal posts were more likely to contain explicit mentions of suicide, descriptions of risk factors (e.g., depression, PTSD) and help-seeking, and first-person singular pronouns.Conclusions: Our results demonstrate the feasibility and potential promise of using social media posts to identify at-risk Servicemembers and Veterans. Future work will use this approach to deliver targeted interventions to social media users at risk for suicide.


  • Open Access

    Building personalized machine learning models using real-time monitoring data to predict idiographic suicidal thoughts

    by Shirley B Wang, Ruben van Genugten, Yaniv Yacoby, Weiwei Pan, Kate Bentley, Suzanne Bird, Ralph Buonopane, Alexis Christie, Merryn Daniel, Dylan DeMarco, et al.

    Suicide risk is highest immediately after psychiatric hospitalization, but the field lacks methods for identifying which patients are at greatest risk, and when. We built personalized models predicting suicidal thoughts after psychiatric hospital visits (N=89 patients), using ecological momentary assessment (EMA; average EMA responses per participant=311). We built several idiographic models, including baseline autoregressive and elastic net models (using single train/test split) and Gaussian Process (GP) models (using an iterative rolling-forward prediction method). Simple GP models provided the best prediction of suicidal urges (R2average=0.17), outperforming baseline autoregressive (R2average=0.10) and elastic net (R2average=0.06) models. Similarly, simple GP models provided the best prediction of suicidal intent (R2average=0.12) compared to autoregressive (R2average=0.08) and elastic net (R2average=0.04). Here we show that idiographic prediction of suicidal thoughts is possible, though accuracy currently is modest. Building GP models that iteratively update and learn symptom dynamics over time could provide important information to inform development of just-in-time adaptive interventions.

  • Open Access

    Characterizing suicidal thinking: Examining momentary and daily motivational states, intent and thought content.

    by Alexander Millner, Daniel D.L. Coppersmith, Rebecca Fortgang, Osiris Rankin, Irene Xu, Merryn Daniel, Kelly L Zuromski, Shirley B Wang, Donald Robinaugh, Kate Bentley, et al.

    Suicide is a leading cause of death. Rich descriptions of suicidal phenomena are critical to advancing understanding. Research on goal-directed behavior suggests that actions require both a motivation and a cognitive representation of the behavior. Therefore, we measured motivational (e.g., desire or urge) and intentional (e.g., intent) suicidal states as well as suicidal thought content (e.g., verbal thoughts, imagery and planning) using momentary (9,786 responses) and daily (2,335 responses) assessment over 42 days in 105 suicidal adults. Results revealed, first that suicidal imagery was more common than verbal suicidal thoughts, and suicide plans co-occurred with imagery approximately 70% of the time. Second, high levels of suicidal desire, urge and intent were frequently accompanied by active suicidal thought content, the reverse was not true: active thought content was frequently present during low or absent motivational/intentional states. These results paint a nuanced and complex picture regarding the phenomenology of suicidal thoughts.


  • Open Access

    Implementing Machine Learning Models for Suicide Risk Prediction in Clinical Practice: Focus Group Study With Hospital Providers

    by Kate H Bentley, Kelly L Zuromski, Rebecca G Fortgang, Emily M Madsen, Daniel Kessler, Hyunjoon Lee, Matthew K Nock, Ben Y Reis, Victor M Castro, Jordan W Smoller

    Background: Interest in developing machine learning models that use electronic health record data to predict patients’ risk of suicidal behavior has recently proliferated. However, whether and how such models might be implemented and useful in clinical practice remain unknown. To ultimately make automated suicide risk–prediction models useful in practice, and thus better prevent patient suicides, it is critical to partner with key stakeholders, including the frontline providers who will be using such tools, at each stage of the implementation process. Objective: The aim of this focus group study is to inform ongoing and future efforts to deploy suicide risk–prediction models in clinical practice. The specific goals are to better understand hospital providers’ current practices for assessing and managing suicide risk; determine providers’ perspectives on using automated suicide risk–prediction models in practice; and identify barriers, facilitators, recommendations, and factors to consider. Methods: We conducted 10 two-hour focus groups with a total of 40 providers from psychiatry, internal medicine and primary care, emergency medicine, and obstetrics and gynecology departments within an urban academic medical center. Audio recordings of open-ended group discussions were transcribed and coded for relevant and recurrent themes by 2 independent study staff members. All coded text was reviewed and discrepancies were resolved in consensus meetings with doctoral-level staff. Results: Although most providers reported using standardized suicide risk assessment tools in their clinical practices, existing tools were commonly described as unhelpful and providers indicated dissatisfaction with current suicide risk assessment methods. Overall, providers’ general attitudes toward the practical use of automated suicide risk–prediction models and corresponding clinical decision support tools were positive. Providers were especially interested in the potential to identify high-risk patients who might be missed by traditional screening methods. Some expressed skepticism about the potential usefulness of these models in routine care; specific barriers included concerns about liability, alert fatigue, and increased demand on the health care system. Key facilitators included presenting specific patient-level features contributing to risk scores, emphasizing changes in risk over time, and developing systematic clinical workflows and provider training. Participants also recommended considering risk-prediction windows, timing of alerts, who will have access to model predictions, and variability across treatment settings. Conclusions: Providers were dissatisfied with current suicide risk assessment methods and were open to the use of a machine learning–based risk-prediction system to inform clinical decision-making. They also raised multiple concerns about potential barriers to the usefulness of this approach and suggested several possible facilitators. Future efforts in this area will benefit from incorporating systematic qualitative feedback from providers, patients, administrators, and payers on the use of these new approaches in routine care, especially given the complex, sensitive, and unfortunately still stigmatized nature of suicide risk.


  • Open Access

    A secondary analysis of a brief video intervention on suicidal ideation among recent rape victims.

    by Kate Walsh, Christal L. Badour, Kelly L. Zuromski, Amanda K. Gilmore, Dean G. Kilpatrick, Ron Acierno, Heidi S. Resnick

    Although rape has been linked to risk for suicidal ideation and attempts, there are scant data on the efficacy of brief interventions to reduce suicidality among recent rape victims. This secondary analysis of a randomized controlled trial investigated whether a video intervention delivered in the emergency department (ED), cortisol measured at the ED, and prior rape history predicted postrape suicidal ideation independently or in combination with the other predictors. Participants were 235 women aged 15-71 years who presented to the ED for a sexual assault medical forensic examination and were randomly assigned to either receive a video intervention that addressed avoidance and promoted healthy coping strategies or standard care prior to the examination. Participants also provided a blood sample for cortisol and completed at least one of three follow-ups at 6 weeks, 3 months, or 6 months postrape. The intervention conferred protection against suicidal ideation among women with elevated cortisol and a prior rape; however, it did not reduce risk for women without a prior rape, particularly those with elevated cortisol. It may be important to consider the influence of prior rape and neuroendocrine reactivity in developing treatments to address suicidal ideation among rape victims. More specifically, there appears to be value in screening victims for prior rape and administering this brief intervention to reduce suicidal ideation; however, other avenues should be explored for victims without a prior rape history. (PsycInfo Database Record (c) 2021 APA, all rights reserved).


  • Open Access

    Implementing machine learning algorithms for suicide risk prediction in clinical practice: A focus group study

    by Kate Bentley, Kelly L Zuromski, Rebecca Fortgang, Emily Madsen, Daniel T. Kessler, Hyunjoon Lee, Matthew Nock, Ben Reis, Victor Castro, Jordan W. Smoller

    Background: Interest in developing machine learning algorithms that use electronic health record data to predict patients’ risk of suicidal behavior has recently proliferated. Whether and how such models might be implemented and useful in clinical practice, however, remains unknown. In order to ultimately make automated suicide risk prediction algorithms useful in practice, and thus better prevent patient suicides, it is critical to partner with key stakeholders (including the frontline providers who will be using such tools) at each stage of the implementation process.Objective: The aim of this focus group study was to inform ongoing and future efforts to deploy suicide risk prediction models in clinical practice. The specific goals were to better understand hospital providers’ current practices for assessing and managing suicide risk; determine providers’ perspectives on using automated suicide risk prediction algorithms; and identify barriers, facilitators, recommendations, and factors to consider for initiatives in this area. Methods: We conducted 10 two-hour focus groups with a total of 40 providers from psychiatry, internal medicine and primary care, emergency medicine, and obstetrics and gynecology departments within an urban academic medical center. Audio recordings of open-ended group discussions were transcribed and coded for relevant and recurrent themes by two independent study staff members. All coded text was reviewed and discrepancies resolved in consensus meetings with doctoral-level staff. Results: Though most providers reported using standardized suicide risk assessment tools in their clinical practices, existing tools were commonly described as unhelpful and providers indicated dissatisfaction with current suicide risk assessment methods. Overall, providers’ general attitudes toward the practical use of automated suicide risk prediction models and corresponding clinical decision support tools were positive. Providers were especially interested in the potential to identify high-risk patients who might be missed by traditional screening methods. Some expressed skepticism about the potential usefulness of these models in routine care; specific barriers included concerns about liability, alert fatigue, and increased demand on the healthcare system. Key facilitators included presenting specific patient-level features contributing to risk scores, emphasizing changes in risk over time, and developing systematic clinical workflows and provider trainings. Participants also recommended considering risk-prediction windows, timing of alerts, who will have access to model predictions, and variability across treatment settings.Conclusions: Providers were dissatisfied with current suicide risk assessment methods and open to the use of a machine learning-based risk prediction system to inform clinical decision-making. They also raised multiple concerns about potential barriers to the usefulness of this approach and suggested several possible facilitators. Future efforts in this area will benefit from incorporating systematic qualitative feedback from providers, patients, administrators, and payers on the use of new methods in routine care, especially given the complex, sensitive, and unfortunately still stigmatized nature of suicide risk.


  • Open Access

    Implementing Machine Learning Models for Suicide Risk Prediction in Clinical Practice: Focus Group Study With Hospital Providers (Preprint)

    by Kate H Bentley, Kelly L Zuromski, Rebecca G Fortgang, Emily M Madsen, Daniel Kessler, Hyunjoon Lee, Matthew K Nock, Ben Y Reis, Victor M Castro, Jordan W Smoller

    BACKGROUND Interest in developing machine learning models that use electronic health record data to predict patients’ risk of suicidal behavior has recently proliferated. However, whether and how such models might be implemented and useful in clinical practice remain unknown. To ultimately make automated suicide risk–prediction models useful in practice, and thus better prevent patient suicides, it is critical to partner with key stakeholders, including the frontline providers who will be using such tools, at each stage of the implementation process. OBJECTIVE The aim of this focus group study is to inform ongoing and future efforts to deploy suicide risk–prediction models in clinical practice. The specific goals are to better understand hospital providers’ current practices for assessing and managing suicide risk; determine providers’ perspectives on using automated suicide risk–prediction models in practice; and identify barriers, facilitators, recommendations, and factors to consider. METHODS We conducted 10 two-hour focus groups with a total of 40 providers from psychiatry, internal medicine and primary care, emergency medicine, and obstetrics and gynecology departments within an urban academic medical center. Audio recordings of open-ended group discussions were transcribed and coded for relevant and recurrent themes by 2 independent study staff members. All coded text was reviewed and discrepancies were resolved in consensus meetings with doctoral-level staff. RESULTS Although most providers reported using standardized suicide risk assessment tools in their clinical practices, existing tools were commonly described as unhelpful and providers indicated dissatisfaction with current suicide risk assessment methods. Overall, providers’ general attitudes toward the practical use of automated suicide risk–prediction models and corresponding clinical decision support tools were positive. Providers were especially interested in the potential to identify high-risk patients who might be missed by traditional screening methods. Some expressed skepticism about the potential usefulness of these models in routine care; specific barriers included concerns about liability, alert fatigue, and increased demand on the health care system. Key facilitators included presenting specific patient-level features contributing to risk scores, emphasizing changes in risk over time, and developing systematic clinical workflows and provider training. Participants also recommended considering risk-prediction windows, timing of alerts, who will have access to model predictions, and variability across treatment settings. CONCLUSIONS Providers were dissatisfied with current suicide risk assessment methods and were open to the use of a machine learning–based risk-prediction system to inform clinical decision-making. They also raised multiple concerns about potential barriers to the usefulness of this approach and suggested several possible facilitators. Future efforts in this area will benefit from incorporating systematic qualitative feedback from providers, patients, administrators, and payers on the use of these new approaches in routine care, especially given the complex, sensitive, and unfortunately still stigmatized nature of suicide risk.