Conversational trajectory degrades large language model detection of suicidal ideation relative to clinicians: a preregistered study
Preprint
Abstract
Background: General-purpose large language models increasingly encounter emotional and therapy-like conversation, yet are not developed or evaluated as clinical systems. Existing safety evaluations rely largely on brief exchanges, although harms often unfold over extended interactions. Whether models maintain safety-relevant performance as conversations accumulate context remains unknown. Methods: In this preregistered study, 400 clinician-validated statements, with or without suicidal ideation, were inserted at 0–200 speaker turns in 5 psychotherapy and 3 synthetic transcripts. Forty-nine LLMs and 8 clinicians performed the same binary classification task. Mixed-effects models estimated the effects of conversational depth, model scale, and model version on F1. Twelve top models were tested to 1,500 turns across conversational trajectories, with or without instruction restatement. Results: F1 declined with depth across model families (p<0.001). Larger, newer models performed better but still degraded. Clinicians showed no decline (mean F1 0.86 at both 0 and 200 turns), but eight of nine proprietary models exceeded their performance at 200 turns. Conversational content, not length alone, explained F1 changes; the largest decrease was under adversarial context (p<0.001). Restating instructions increased F1 on therapy to near baseline (median ΔF1 +0.12; p<0.001; 89% median recovery) versus MSJ (ΔF1 +0.08; p=0.04; 38% recovery). Conclusions: LLM detection of suicidal ideation degraded with conversational depth and trajectory, whereas clinician performance remained stable despite the strongest models exceeding most clinicians in absolute performance. Mental health AI safety evaluations should test sustained performance across realistic and adversarial trajectories rather than relying on short-prompt benchmarks. Plain-language summary: Large language models increasingly encounter psychiatric distress and crisis-related content in ordinary use, but existing mental health AI safety benchmarks often rely on isolated prompts, brief exchanges, or simulated conversations rather than real extended interactions. In a preregistered benchmark of 1.4M inferences from 49 LLMs using real psychotherapy transcripts, suicidal-ideation detection declined with conversational depth in every model family while clinician performance remained stable — despite the strongest models matching or exceeding most clinicians in absolute performance — and, among top models, degradation at extended depths depended on the content and direction of the preceding conversation rather than length alone.Keywords
This publication has 17 references indexed in Scilit:
- Complexity of Diagnosing Volume Overload—ReplyJAMA, 2026
- Leveraging simulation to provide a practical framework for estimating the novel scope of risk of large language models in healthcareBMJ Mental Health, 2026
- Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial IntelligenceIEEE Transactions on Artificial Intelligence, 2025
- A scoping review of large language models for generative tasks in mental health carenpj Digital Medicine, 2025
- Benchmark evaluation of DeepSeek large language models in clinical decision-makingNature Medicine, 2025
- Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks2025
- The health risks of generative AI-based wellness appsNature Medicine, 2024
- Lost in the Middle: How Language Models Use Long ContextsTransactions of the Association for Computational Linguistics, 2024
- Evidence for Anchoring Bias During Physician Decision-MakingJAMA Internal Medicine, 2023
- The Columbia–Suicide Severity Rating Scale: Initial Validity and Internal Consistency Findings From Three Multisite Studies With Adolescents and AdultsAmerican Journal of Psychiatry, 2011