Scaling Trends for Data Poisoning in LLMs
Open Access
- 11 April 2025
- journal article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (26) , 27206-27214
Abstract
LLMs produce harmful and undesirable behavior when trained on datasets containing even a small fraction of poisoned data. We demonstrate that GPT models remain vulnerable to fine-tuning on poisoned data, even when safeguarded by moderation systems. Given the persistence of data poisoning vulnerabilities in today's most capable models, this paper investigates whether these risks increase with model scaling. We evaluate three threat models—malicious fine-tuning, imperfect data curation, and intentional data contamination—across 24 frontier LLMs ranging from 1.5 to 72 billion parameters. Our experiments reveal that larger LLMs are significantly more susceptible to data poisoning, learning harmful behaviors from even minimal exposure to harmful data more quickly than smaller models. These findings underscore the need for leading AI companies to thoroughly red team fine-tuning APIs before public release and to develop more robust safeguards against data poisoning, particularly as models continue to scale in size and capability.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Poisoning Web-Scale Training Datasets is Practical2024
- PoisonPrompt: Backdoor Attack on Prompt-Based Large Language Models2024
- Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection2024
- Inference on WinnersThe Quarterly Journal of Economics, 2023
- AI model GPT-3 (dis)informs us better than humansScience Advances, 2023
- QLoRA: Efficient Finetuning of Quantized LLMs2023
- Multiple Inference: A Python package for comparing multiple parametersJournal of Open Source Software, 2022
- Revisiting Neural Scaling Laws in Language and Vision2022
- Hidden Trigger Backdoor AttacksProceedings of the AAAI Conference on Artificial Intelligence, 2020
- On defending against label flipping attacks on malware detection systemsNeural Computing and Applications, 2020