Beyond Benchmarks: Dynamic, Automatic And Systematic Red-Teaming Agents For Trustworthy Medical Language Models
Preprint
- 18 February 2026
- preprint
- Published by Springer Nature in Research Square
Abstract
Ensuring the safety and reliability of large language models (LLMs) in clinical practice is critical to prevent patient harm. However, LLMs are advancing so rapidly that static benchmarks quickly become obsolete or prone to overfitting, yielding a misleading picture of model trustworthiness. Here we introduce a Dynamic, Automatic, and Systematic (DAS) red-teaming framework that continuously stress-tests LLMs across four safety-critical axes: robustness, privacy, bias/fairness, and hallucination. Validated against board-certified clinicians with high concordance, a suite of adversarial agents autonomously mutates clinical test cases to uncover vulnerabilities in real time. Applying DAS to 15 proprietary and open-source LLMs revealed a profound gap between high static benchmark performance and low dynamic reliability - the ''Benchmarking Gap''. Despite median MedQA accuracy exceeding 80%, 94% of previously correct answers failed our dynamic robustness tests. Crucially, this brittleness generalized to the realistic, open-ended HealthBench dataset, where top-tier models exhibited failure rates exceeding 70% and stark shifts in model rankings across evaluations, suggesting that high scores on established static benchmarks may reflect superficial memorization. We observed similarly high failure rates across other domains: privacy leaks were elicited in 86% of scenarios, cognitive-bias priming altered clinical recommendations in 81% of fairness tests, and we identified hallucination rates exceeding 74% in widely used models. By converting medical LLM safety analysis from a static checklist into a dynamic stress-test, DAS provides a foundational, scalable, and living platform to surface the latent risks that must be addressed before the next generation of medical AI can be safely deployed.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- Holistic evaluation of large language models for medical tasks with MedHELMNature Medicine, 2026
- MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning2025
- Fidelity of Medical Reasoning in Large Language ModelsJAMA Network Open, 2025
- MedVH: Toward Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical ContextAdvanced Intelligent Systems, 2025
- Towards accurate differential diagnosis with large language modelsNature, 2025
- Sociodemographic biases in medical decision making by large language modelsNature Medicine, 2025
- Toward expert-level medical question answering with large language modelsNature Medicine, 2025
- A generalist vision–language foundation model for diverse biomedical tasksNature Medicine, 2024
- Evaluation and mitigation of the limitations of large language models in clinical decision-makingNature Medicine, 2024
- Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation studyThe Lancet Digital Health, 2023