Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting
Preprint
Abstract
LLM-based agents are increasingly deployed in multi-agent systems (MAS). As these systems move toward real-world applications, their security becomes paramount. Existing research largely evaluates single-agent security, leaving a critical gap in understanding the vulnerabilities introduced by multi-agent design. However, existing systems fall short due to lack of unified frameworks and metrics focusing on unique rejection modes in MAS. We present SafeAgents, a unified and extensible framework for fine-grained security assessment of MAS. SafeAgents systematically exposes how design choices such as plan construction strategies, inter-agent context sharing, and fallback behaviors affect susceptibility to adversarial prompting. We introduce Dharma, a diagnostic measure that helps identify weak links within multi-agent pipelines. Using SafeAgents, we conduct a comprehensive study across five widely adopted multi-agent architectures (centralized, decentralized, and hybrid variants) on four datasets spanning web tasks, tool use, and code generation. Our findings reveal that common design patterns carry significant vulnerabilities. For example, centralized systems that delegate only atomic instructions to sub-agents obscure harmful objectives, reducing robustness. Our results highlight the need for security-aware design in MAS. Link to code is https://github.com/microsoft/SafeAgentsKeywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Multi-Agent Security Tax: Trading Off Security and Collaboration Capabilities in Multi-Agent SystemsProceedings of the AAAI Conference on Artificial Intelligence, 2025
- AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection2025
- WildGuard: Open One-stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs2024
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models2024
- RedCode: Risky Code Execution and Generation Benchmark for Code Agents2024
- MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate2024
- InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents2024
- Jailbroken: How Does LLM Safety Training Fail?2023
- Are aligned neural networks adversarially aligned?2023
- Training Language Models to Follow Instructions with Human Feedback2022