A Pre-mortem on The First Major AI-Enabled Terrorism Event and The Choices That Could Enable It (Plus 9 Predictions)

Authors:

Abstract
Frontier AI developers' model reports and the dedicated risk-assessment teams at governmental AI institutes such as UK AISI and US CAISSI now treat a major AI-enabled cyber, biological, or chemical terror event as a credible concern. But how, precisely, might such an event happen? What choices across the model lifecycle may contribute to it? This paper studies what current evidence suggests about how specific mistakes from AI developers, distributors, and governments may contribute to such an event. In doing so, we consider current research on frontier model risks as well as existing examples of how AI capabilities useful for criminal activities have already been introduced and spread. We make two arguments. Firstly, specific developer, distributor, and government choices around model deployment, safeguards, and detection strategies will substantially shape whether AI meaningfully lowers the barrier to such an attack. Secondly, if a major AI-enabled cyber, biological, or chemical terror event occurs in the foreseeable future, it will most likely have been enabled either (1) as the consequence of a closed-weight model deployed without state-of-the-art safeguards or (2) as a consequence of an open-weight model whose safeguards (if significant ones exist) could not survive downstream tampering. Finally, we conclude with a set of 9 falsifiable predictions about how the first major AI-enabled cyber, bio, or chemical terrorism attack may occur and outline two illustrative hypothetical scenarios for such an event.