On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling
Preprint
Abstract
Today's text-to-image generative models are trained on millions of images sourced from the Internet, each paired with a detailed caption produced by Vision-Language Models (VLMs). This part of the training pipeline is critical for supplying the models with large volumes of high-quality image-caption pairs during training. However, recent work suggests that VLMs are vulnerable to stealthy adversarial attacks, where adversarial perturbations are added to images to mislead the VLMs into producing incorrect captions. In this paper, we explore the feasibility of adversarial mislabeling attacks on VLMs as a mechanism to poisoning training pipelines for text-to-image models. Our experiments demonstrate that VLMs are highly vulnerable to adversarial perturbations, allowing attackers to produce benign-looking images that are consistently miscaptioned by the VLM models. This has the effect of injecting strong "dirty-label" poison samples into the training pipeline for text-to-image models, successfully altering their behavior with a small number of poisoned samples. We find that while potential defenses can be effective, they can be targeted and circumvented by adaptive attackers. This suggests a cat-and-mouse game that is likely to reduce the quality of training data and increase the cost of text-to-image model development. Finally, we demonstrate the real-world effectiveness of these attacks, achieving high attack success (over 73%) even in black-box scenarios against commercial VLMs (Google Vertex AI and Microsoft Azure).Keywords
All Related Versions
- Version 1, 2025-06-27, ArXiv
- Published version: , 2848.
This publication has 8 references indexed in Scilit:
- BLIP-3: A Family of Open Large Multimodal Models2025
- InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks2025
- AI models collapse when trained on recursively generated dataNature, 2024
- Visual Adversarial Examples Jailbreak Aligned Large Language ModelsProceedings of the AAAI Conference on Artificial Intelligence, 2024
- VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models2023
- LAION-5B: An Open Large-Scale Dataset for Training Next Generation Image-Text Models2022
- One-to-N & N-to-One: Two Advanced Backdoor Attacks Against Deep Learning ModelsIEEE Transactions on Dependable and Secure Computing, 2020
- BadNets: Evaluating Backdooring Attacks on Deep Neural NetworksIEEE Access, 2019