Attacking Vision-Language Computer Agents via Pop-ups
Preprint
Abstract
Autonomous agents powered by large vision and language models (VLM) have demonstrated significant potential in completing daily computer tasks, such as browsing the web to book travel and operating desktop software, which requires agents to understand these interfaces. Despite such visual inputs becoming more integrated into agentic applications, what types of risks and attacks exist around them still remain unclear. In this work, we demonstrate that VLM agents can be easily attacked by a set of carefully designed adversarial pop-ups, which human users would typically recognize and ignore. This distraction leads agents to click these pop-ups instead of performing the tasks as usual. Integrating these pop-ups into existing agent testing environments like OSWorld and VisualWebArena leads to an attack success rate (the frequency of the agent clicking the pop-ups) of 86% on average and decreases the task success rate by 47%. Basic defense techniques such as asking the agent to ignore pop-ups or including an advertisement notice, are ineffective against the attack.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions2025
- MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI2024
- Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents2024
- VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks2024
- How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs2024
- Detection of cross-site scripting (XSS) attacks using machine learning techniques: a reviewArtificial Intelligence Review, 2023
- Threat Analysis of Fake Virus Alerts Using WebView Monitor2019
- Perceptual representation of spam and phishing emailsApplied Cognitive Psychology, 2019
- Current state of research on cross-site scripting (XSS) – A systematic literature reviewInformation and Software Technology, 2015