A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks
Authors:
- 21 December 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 207-212
Abstract
Prompt injection attacks represent a major vulnerability in Large Language Model (LLM) deployments, where malicious instructions embedded in user inputs can override system prompts and induce unintended behaviors. This paper presents a novel multi-agent defense framework that employs specialized LLM agents in coordinated pipelines to detect and neutralize prompt injection attacks in real-time. We evaluate our approach using two distinct architectures: a sequential chain-ofagents pipeline and a hierarchical coordinator-based system. Our comprehensive evaluation on 55 unique prompt injection attacks, grouped into 8 categories and totaling 400 attack instances across two LLM platforms (ChatGLM and Llama2), demonstrates significant security improvements. Without defense mechanisms, baseline Attack Success Rates (ASR) reached $\mathbf{3 0 \%}$ for ChatGLM and 20 % for Llama2. Our multi-agent pipeline achieved 100 % mitigation, reducing ASR to 0 % across all tested scenarios. The framework demonstrates robustness across multiple attack categories including direct overrides, code execution attempts, data exfiltration, and obfuscation techniques, while maintaining system functionality for legitimate queries.
Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Enhancing System Security: LLM-Driven Defense Against Prompt Injection Vulnerabilities2024
- Defending ChatGPT against jailbreak attack via self-remindersNature Machine Intelligence, 2023
- A Novel Evaluation Framework for Assessing Resilience Against Prompt Injection Attacks in Large Language Models2023
- Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection2023
- Are aligned neural networks adversarially aligned?2023
- Multi-step Jailbreaking Privacy Attacks on ChatGPT2023
- Training Language Models to Follow Instructions with Human Feedback2022
- Universal Adversarial Triggers for Attacking and Analyzing NLP2019