BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text
Authors:
Preprint
Abstract
Large language models (LLMs) hold great promise for medical applications and are evolving rapidly, with new models being released at an accelerated pace. However, current evaluations of LLMs in clinical contexts remain limited. Most existing benchmarks rely on medical exam-style questions or PubMed-derived text, failing to capture the complexity of real-world electronic health record (EHR) data. Others focus narrowly on specific application scenarios, limiting their generalizability across broader clinical use. To address this gap, we present BRIDGE, a comprehensive multilingual benchmark comprising 87 tasks sourced from real-world clinical data sources across nine languages. We systematically evaluated 52 state-of-the-art LLMs (including DeepSeek-R1, GPT-4o, Gemini, and Llama 4) under various inference strategies. With a total of 13,572 experiments, our results reveal substantial performance variation across model sizes, languages, natural language processing tasks, and clinical specialties. Notably, we demonstrate that open-source LLMs can achieve performance comparable to proprietary models, while medically fine-tuned LLMs based on older architectures often underperform versus updated general-purpose models. The BRIDGE and its corresponding leaderboard serve as a foundational resource and a unique reference for the development and evaluation of new LLMs in real-world clinical text understanding.Keywords
All Related Versions
This publication has 63 references indexed in Scilit:
- Comparison of Frontier Open-Source and Proprietary Large Language Models for Complex DiagnosesJAMA Health Forum, 2025
- Testing and Evaluation of Health Care Applications of Large Language ModelsJAMA, 2025
- Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and FormatJAMA Network Open, 2024
- Assessing Biases in Medical Decisions via Clinician and AI Chatbot Responses to Patient VignettesJAMA Network Open, 2023
- Accuracy and Reliability of Chatbot Responses to Physician QuestionsJAMA Network Open, 2023
- Evaluating the Application of Large Language Models in Clinical Research ContextsJAMA Network Open, 2023
- Creation and Adoption of Large Language Models in MedicineJAMA, 2023
- Large Language Models Answer Medical Questions Accurately, but Can’t Match Clinicians’ KnowledgeJAMA, 2023
- Comparison of History of Present Illness Summaries Generated by a Chatbot and Senior Internal Medicine ResidentsJAMA Internal Medicine, 2023
- The Challenges for Regulating Medical Use of ChatGPT and Other Large Language ModelsJAMA, 2023