Andrew Lohn

Background: Generative artificial intelligence (GAI) chatbots are frequently used for mental health–related questions, including suicide-related queries. User interfaces (UIs) may include additional safety controls not present in direct application programming interface (API) access (eg, moderation rules that flag or block harmful content, crisis refusal templates, or system-level instructions). Objective: This observational cross-sectional study aimed to examine how frequently AI chatbots provide direct responses to suicide-related prompts across access modes and whether response patterns differ between consumer-facing UI and API access across models and prompt risk levels. Methods: We evaluated how five widely used consumer AI models (ChatGPT, Claude, Gemini, Grok, and Llama) responded to 30 previously vetted suicide-related prompts spanning five clinician-assigned risk levels. Each prompt was submitted 100 times through both public-facing UI and direct API access, yielding 30,000 total responses. The primary outcome was whether a response directly answered the prompt (ie, provided specific information or guidance related to the question asked). AI responses were categorized using a blinded large language model–based classifier (GPT-4o mini), validated against independent human coding (289/300, 96.3% agreement; Cohen κ=0.89). We estimated mixed-effects logistic regression models that predicted a direct response. The primary predictors were the AI model, access mode (UI vs API), and prompt risk category. Results: Of the 30,000 responses, 69.8% (n=20,934) were direct. Direct responses were more common through APIs than UIs (n=11,609, 77.4%, vs n=9325, 62.2%); however, Gemini had slightly higher direct response rates via UIs than APIs (n=2301, 76.7%, vs n=2131, 71.0%). Differences in the likelihood of a direct response were most pronounced for higher-risk prompts: For very-high-risk prompts, 24.8% (n=743) of API responses were direct compared with 4.6% (n=138) of UI responses, whereas for high-risk prompts, 80.1% (n=2002) of API responses were direct compared with 48.4% (n=1209) of UI responses. Claude and Gemini had the highest direct response rates (n=4688, 78.1%, and n=4432, 73.9%, respectively) across UI and API access modes, whereas ChatGPT, Grok, and Llama had lower rates (n=3878, 64.6%; n=3870, 64.5%; and n=4066, 67.8%, respectively). In mixed-effects models, UI access was associated with lower odds of a direct response than API access (odds ratio 0.09, 95% CI 0.08-0.10). Higher prompt risk was associated with lower direct response probability, and access mode differences varied by risk level and model. Conclusions: AI safety depends on how users access the model. Safety observed in a UI should not be assumed to generalize to API access or apps built on APIs. Evaluations and policies should consider access modes to ensure safety protections for individuals interacting with AI chatbots.

Winning the AI race does not guarantee any economic, social, or military benefit. Without a theory of victory mapping AI development and deployment to competitive advantage, and ultimately to beneficial outcomes like profit or military victory, AI might provide more impressive capability than tangible benefit. This report evaluates various types of advantages that winning the AI race might provide and describes six theories of victory for converting those advantages into worthwhile outcomes.

As artificial intelligence introduces new risks, some potentially catastrophic or even existential, there is little data or detailed theory to assess them. Policymakers often resort to expert best guesses for the probability of doom but probability is not always the most appropriate tool, especially for the types of uncertainties in AI risk. This report details a brief introduction to Belief and Plausibility, which provides an alternative approach that is mathematically rigorous, uses familiar vocabulary, and only requires policymakers to ask two simple questions.

BACKGROUND Generative AI chatbots are frequently used for mental health-related questions, including suicide-related queries. Prior evaluations have focused on user interfaces (UI), which may include additional safety controls not present in direct application programming interface (API) access. OBJECTIVE To examine how frequently AI chatbot models provide direct responses to suicide-related prompts across access pathways and to determine whether safer behavior is intrinsic to the model or depends on UI-level safeguards. METHODS Observational cross-sectional study conducted in February and March 2026. We evaluated how five widely used consumer AI models (ChatGPT, Claude, Gemini, Grok, and Llama) responded to 30 previously vetted suicide-related prompts spanning five clinician-assigned risk levels. Each prompt was submitted 100 times through both public-facing UI and direct API access, yielding 30,000 total responses. The primary outcome was whether a response directly answered the suicide-related prompt. A direct response was defined as one that provided specific information or guidance related to the question asked, rather than refusing to answer, redirecting the user to a crisis resource, or responding only with general safety language. AI responses were categorized using a blinded large language model–based classifier. We estimated mixed-effects logistic regression models that predicted a direct response. The primary predictors were the AI model, access mode (UI vs. API), and prompt risk category. RESULTS 69.8% of responses to the suicide prompts were direct. Direct responses were more common through APIs than UIs (77.4% vs. 62.2%). Differences in the likelihood of a direct response were most pronounced for higher-risk prompts: among very high-risk prompts, 24.8% of API responses were direct compared with 4.6% of UI responses; among high-risk prompts, 80.1% of API responses were direct compared with 48.4% of UI responses. Claude and Gemini had the highest direct response rates (78.1% and 73.9%), whereas ChatGPT, Grok, and Llama had lower rates (64.6%, 64.5%, and 67.8%). In mixed-effects models, UI access was associated with lower odds of a direct response than API access (odds ratio, 0.09; 95% CI, 0.08-0.10). Higher prompt risk was associated with lower direct response probability, and access-mode differences varied by risk level and model. CONCLUSIONS AI safety behavior depends on how the user accesses the model. The safety observed in an AI chatbot’s UI should not be assumed to generalize direct model access through APIs or to downstream applications built on those APIs. Evaluations and policies must consider access channels to ensure comprehensive safety protections for all individuals interacting with AI chatbots.

Artificial intelligence (AI) is beginning to change cybersecurity. This report takes a comprehensive look across cybersecurity to anticipate whether those changes will help cyber defense or offense. Rather than a single answer, there are many ways that AI will help both cyber attackers and defenders. The report finds that there are also several actions that defenders can take to tilt the odds to their favor.

Computer scientists have long sought to build systems that can actively and autonomously carry out complicated goals in the real world—commonly referred to as artificial intelligence "agents." Recently, significant progress in large language models has fueled new optimism about the prospect of building sophisticated AI agents. This CSET-led workshop report synthesizes findings from a May 2024 workshop on this topic, including what constitutes an AI agent, how the technology is improving, what risks agents exacerbate, and intervention points that could help.

Concerns over risks from generative artificial intelligence systems have increased significantly over the past year, driven in large part by the advent of increasingly capable large language models. But, how do AI developers attempt to control the outputs of these models? This primer outlines four commonly used techniques and explains why this objective is so challenging.

While recent progress in artificial intelligence (AI) has relied primarily on increasing the size and scale of the models and computing budgets for training, we ask if those trends will continue. Financial incentives are against scaling, and there can be diminishing returns to further investment. These effects may already be slowing growth among the very largest models. Future progress in AI may rely more on ideas for shrinking models and inventive use of existing models than on simply increasing investment in compute resources.

Place to find CSET's publications, reports, and people

The current AI-for-cybersecurity paradigm focuses on detection using automated tools, but it has largely neglected holistic autonomous cyber defense systems — ones that can act without human tasking. That is poised to change as tools are proliferating for training reinforcement learning-based AI agents to provide broader autonomous cybersecurity capabilities. The resulting agents are still rudimentary and publications are few, but the current barriers are surmountable and effective agents would be a substantial boon to society.