Addressing benchmarking gaps in large language models for health and medicine with dynamic red-teaming
Authors:
- Jiazhen PanJPJiazhen Pan
- Bailiang JianBJBailiang Jian
- Paul HagerPHPaul Hager
- Yundi ZhangYZYundi Zhang
- Che LiuCLChe Liu
- Friedrike JungmannFJFriedrike Jungmann
- Hongwei Bran LiHLHongwei Bran Li
- Julian CanisiusJCJulian Canisius
- Chenyu YouCYChenyu You
- Junde WuJWJunde Wu
- Jiayuan ZhuJZJiayuan Zhu
- Fenglin LiuFLFenglin Liu
Open Access
- 15 July 2026
- journal article
- Published by Springer Nature in Nature Health
- Vol. 1 (11) , 1254-1270
Abstract
Large language models (LLMs) are increasingly used to answer health-related questions and support healthcare workflows, yet evidence for their safety still relies heavily on static benchmarks that can rapidly become obsolete or be optimized against. Here we introduce a Dynamic, Automatic and Systematic (DAS) red-teaming audit framework that continuously stress-tests LLMs for health across four safety-critical axes: robustness, privacy, bias and hallucination. Validated against board-certified clinicians, a suite of adversarial agents autonomously mutates health-related test cases to uncover vulnerabilities in real time. Applying DAS to 15 state-of-the-art LLMs revealed a profound gap between high static benchmark performance and low dynamic reliability—the ‘benchmarking gap’. Despite median MedQA accuracy exceeding 80%, 94% of previously correct answers failed under dynamic robustness testing. This brittleness generalized to the realistic, open-ended HealthBench dataset, where top-tier models exhibited failure rates exceeding 70%, suggesting that high scores on established static benchmarks may reflect superficial memorization. We observed similarly high failure rates across other domains: privacy leaks were elicited in 86% of scenarios, cognitive bias priming altered recommendations in 81% of fairness tests and hallucination rates exceeded 74% in widely used models. By converting LLM safety evaluation for health from a static checklist into a living adversarial audit, DAS provides a scalable framework for surfacing latent risks before such systems are deployed in consumer-facing health assistants and broader clinical workflows.Keywords
This publication has 39 references indexed in Scilit:
- MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning2025
- Fidelity of Medical Reasoning in Large Language ModelsJAMA Network Open, 2025
- MedVH: Toward Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical ContextAdvanced Intelligent Systems, 2025
- Towards accurate differential diagnosis with large language modelsNature, 2025
- Large Language Models for Chatbot Health Advice StudiesJAMA Network Open, 2025
- A generalist vision–language foundation model for diverse biomedical tasksNature Medicine, 2024
- Evaluation and mitigation of the limitations of large language models in clinical decision-makingNature Medicine, 2024
- A visual-language foundation model for computational pathologyNature Medicine, 2024
- Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation studyThe Lancet Digital Health, 2023
- Problems of Monetary Management: The UK Experience1984