BRIDGE: benchmarking large language models for understanding real-world clinical practice texts
Top Cited Papers
- 17 June 2026
- journal article
- research article
- Published by Springer Nature in Nature Biomedical Engineering
- p. 1-16
Abstract
Large language models (LLMs) are evolving rapidly and hold great promise for medical applications, yet benchmarking on real-world clinical data such as electronic health records remains limited. Most existing benchmarks rely on medical examination-style questions or PubMed-derived text, failing to capture the complexity of clinical practice, while others target specific application scenarios with limited generalizability. Here we present BRIDGE, a comprehensive multilingual benchmark comprising 87 tasks sourced from 59 real-world clinical data sources across 9 languages. It covers eight task types spanning the patient care continuum, including triage, information extraction, diagnosis, prognosis and billing coding, and involves 14 clinical specialties. We systematically evaluated 95 LLMs (including DeepSeek-R1, GPT-4o, Gemini and Qwen3) under multiple inference strategies. Results reveal substantial performance variation across model sizes, languages, natural language processing tasks and clinical specialties. Open-source LLMs can match proprietary models, while medically fine-tuned models built on older backbones often lag behind updated general-purpose LLMs. BRIDGE and its continuously updated leaderboard provide foundational resources and important references for evaluating and developing LLMs for real-world clinical text understanding.Keywords
All Related Versions
Funding Information
- Patient-Centered Outcomes Research Institute (ME-2022C1-25646)
- National Institute on Aging (RF1AG090405)
- U.S. National Library of Medicine (R01LM014667)
- Brigham Research Institute
This publication has 82 references indexed in Scilit:
- Comparison of Frontier Open-Source and Proprietary Large Language Models for Complex DiagnosesJAMA Health Forum, 2025
- Testing and Evaluation of Health Care Applications of Large Language ModelsJAMA, 2025
- Large Language Model Influence on Diagnostic ReasoningJAMA Network Open, 2024
- Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and FormatJAMA Network Open, 2024
- Diagnostic Accuracy of a Large Language Model in Pediatric Case StudiesJAMA Pediatrics, 2024
- Assessing Biases in Medical Decisions via Clinician and AI Chatbot Responses to Patient VignettesJAMA Network Open, 2023
- Accuracy and Reliability of Chatbot Responses to Physician QuestionsJAMA Network Open, 2023
- Creation and Adoption of Large Language Models in MedicineJAMA, 2023
- Comparison of History of Present Illness Summaries Generated by a Chatbot and Senior Internal Medicine ResidentsJAMA Internal Medicine, 2023
- The Challenges for Regulating Medical Use of ChatGPT and Other Large Language ModelsJAMA, 2023