Medical large language models are susceptible to targeted misinformation attacks
Open Access
- 23 October 2024
- journal article
- research article
- Published by Springer Nature in npj Digital Medicine
- Vol. 7 (1) , 1-9
Abstract
Large language models (LLMs) have broad medical knowledge and can reason about medical information across many domains, holding promising potential for diverse medical applications in the near future. In this study, we demonstrate a concerning vulnerability of LLMs in medicine. Through targeted manipulation of just 1.1% of the weights of the LLM, we can deliberately inject incorrect biomedical facts. The erroneous information is then propagated in the model’s output while maintaining performance on other biomedical tasks. We validate our findings in a set of 1025 incorrect biomedical facts. This peculiar susceptibility raises serious security and trustworthiness concerns for the application of LLMs in healthcare settings. It accentuates the need for robust protective measures, thorough verification mechanisms, and stringent management of access to these models, ensuring their reliable and safe use in medical practice.Keywords
All Related Versions
This publication has 42 references indexed in Scilit:
- Comparative Analysis of Multimodal Large Language Model Performance on Clinical Vignette QuestionsJAMA, 2024
- The impact of ChatGPT and LLMs on medical imaging stakeholders: Perspectives and use casesMeta-Radiology, 2023
- Foundation models for generalist medical artificial intelligenceNature, 2023
- Catalyzing next-generation Artificial Intelligence through NeuroAINature Communications, 2023
- ChatGPT: five priorities for researchNature, 2023
- Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learningNature Biomedical Engineering, 2022
- Self-supervised learning in medicine and healthcareNature Biomedical Engineering, 2022
- Towards artificial general intelligence via a multimodal foundation modelNature Communications, 2022
- Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalizationNature Communications, 2021
- Aspirin as a Risk Factor in Reye's SyndromeJAMA, 1982