ZAZaid Al-Fagih

Zaid Al-Fagih

*This is an algorithmically generated author record based on publicly available information.

Zaid Al-Fagih has a total of 8 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Challenges and Solutions in Applying Large Language Models to Guideline-Based Management Planning and Automated Medical Coding in Health Care: Algorithm Development and Validation

    by Peter Sarvari, Zaid Al-Fagih, Alexander Abou-Chedid, Paul Jewell, Rosie Taylor, Arouba Imtiaz

    Background: Diagnostic errors and administrative burdens, including medical coding, remain major challenges in health care. Large language models (LLMs) have the potential to alleviate these problems, but their adoption has been limited by concerns regarding reliability, transparency, and clinical safety. Objective: This study introduces and evaluates 2 LLM-based frameworks, implemented within the Rhazes Clinician platform, designed to address these challenges: generation-assisted retrieval-augmented generation (GARAG) for automated evidence-based treatment planning and generation-assisted vector search (GAVS) for automated medical coding. Methods: GARAG was evaluated on 21 clinical test cases created by medically qualified authors. Each case was executed 3 times independently, and outputs were assessed using 4 criteria: correctness of references, absence of duplication, adherence to formatting, and clinical appropriateness of the generated management plan. GAVS was evaluated on 958 randomly selected admissions from the Medical Information Mart for Intensive Care (MIMIC)–IV database, in which billed International Classification of Diseases, Tenth Revision ( ICD-10 ) codes served as the ground truth. Two approaches were compared: a direct GPT-4.1 baseline prompted to predict ICD-10 codes without constraints and GAVS, in which GPT-4.1 generated diagnostic entities that were each mapped onto the top 10 matching ICD-10 codes through vector search. Results: Across the 63 outputs, 62 (98.4%) satisfied all evaluation criteria, with the only exception being a minor ordering inconsistency in one repetition of case 14. For GAVS, the 958 admissions contained 8576 assigned ICD-10 subcategory codes (1610 unique). The vanilla LLM produced 131,329 candidate codes, whereas GAVS produced 136,920. At the subcategory level, the vanilla LLM achieved 17.95% average recall (15.86% weighted), while GAVS achieved 20.63% (18.62% weighted), a statistically significant improvement ( P <.001). At the category level, performance converged (32.60% vs 32.58% average weighted recall; P =.99). Conclusions: GARAG demonstrated a workflow that grounds management plans in diagnosis-specific, peer-reviewed guideline evidence, preserving fine-grained clinical detail during retrieval. GAVS significantly improved fine-grained diagnostic coding recall compared with a direct LLM baseline. Together, these frameworks illustrate how LLM-based methods can enhance clinical decision support and medical coding. Both were subsequently integrated into Rhazes Clinician, a clinician-facing web application that orchestrates LLM agents to call specialized tools, providing a single interface for physician use. Further independent validation and large-scale studies are required to confirm generalizability and assess their impact on patient outcomes.


  • Open Access

    Authors’ Response to Peer Reviews of “Rapidly Benchmarking Large Language Models for Diagnosing Comorbid Patients: Comparative Study Leveraging the LLM-as-a-Judge Method”

    by Peter Sarvari, Zaid Al-Fagih

    This is the author(s)' response to peer review reports related to "Rapidly benchmarking Large Language Models for diagnosing comorbid patients: A comparative study leveraging the LLM-as-a-judge method"

  • Open Access

    Rapidly Benchmarking Large Language Models for Diagnosing Comorbid Patients: Comparative Study Leveraging the LLM-as-a-Judge Method

    by Peter Sarvari, Zaid Al-Fagih

    Background: On average, 1 in 10 patients die because of a diagnostic error, and medical errors represent the third largest cause of death in the United States. While large language models (LLMs) have been proposed to aid doctors in diagnoses, no research results have been published comparing the diagnostic abilities of many popular LLMs on a large, openly accessible real-patient cohort. Objective: In this study, we set out to compare the diagnostic ability of 18 LLMs from Google, OpenAI, Meta, Mistral, Cohere, and Anthropic, using 3 prompts, 2 temperature settings, and 1000 randomly selected Medical Information Mart for Intensive Care-IV (MIMIC-IV) hospital admissions. We also explore improving the diagnostic hit rate of GPT-4o 05‐13 with retrieval-augmented generation (RAG) by utilizing reference ranges provided by the American Board of Internal Medicine. Methods: We evaluated the diagnostic ability of 21 LLMs, using an LLM-as-a-judge approach (an automated, LLM-based evaluation) on MIMIC-IV patient records, which contain final diagnostic codes. For each case, a separate assessor LLM (“judge”) compared the predictor LLM’s diagnostic output to the true diagnoses from the patient record. The assessor determined whether each true diagnosis was inferable from the available data and, if so, whether it was correctly predicted (“hit”) or not (“miss”). Diagnoses not inferable from the patient record were excluded from the hit rate analysis. The reported hit rate was defined as the number of hits divided by the total number of hits and misses. The statistical significance of the differences in model performance was assessed using a pooled z-test for proportions. Results: Gemini 2.5 was the top performer with a hit rate of 97.4% (95% CI 97.0%‐97.8%) as assessed by GPT-4.1, significantly outperforming GPT-4.1, Claude-4 Opus, and Claude Sonnet. However, GPT-4.1 ranked the highest in a separate set of experiments evaluated by GPT-4 Turbo, which tended to be less conservative than GPT-4.1 in its assessments. Significant variation in diagnostic hit rates was observed across different prompts, while changes in temperature generally had little effect. Finally, RAG significantly improved the hit rate of GPT-4o 05‐13 by an average of 0.8% (P<.006). Conclusions: While the results are promising, more diverse datasets and hospital pilots, as well as close collaborations with physicians, are needed to obtain a better understanding of the diagnostic abilities of these models.


  • Open Access

    Towards Evaluating the Diagnostic Ability of LLMs

    by Peter Sarvari, Zaid Al-Fagih

    On average, one in ten patients die because of a diagnostic error and medical errors are the third largest cause of death in the US. While LLMs have been proposed to help doctors with diagnoses, no research results have been published on comparing the diagnostic ability of many popular LLMs on an openly accessible real-patient cohort. In this study, we compare LLMs from Google, OpenAI, Meta, Mistral, Cohere and Anthropic using a previously established evaluation methodology and explore improving their accuracy with RAG. We found that GPT-4o from OpenAI and Claude Sonnet 3.5 from Anthropic were the top performers with them only missing 0.5% of ground truth conditions that were clearly inferable from the available data; RAG further improved this error rate to 0.2%. While the results are promising, more diverse datasets, hospital pilots and close collaboration with physicians are needed to get a better understanding of the diagnostic ability of these models.

  • Open Access

    Rhazes: An AI-Assistant to help with the paperwork and analytical tasks in clinical medicine (Preprint)

    by Peter Sarvari, Zaid Al-Fagih, Alexander Abou-Chedid

    BACKGROUND One in ten patients die because of a diagnostic error while doctors spend two hours with paperwork for every hour they spend with patients. Medical knowledge base doubles every 73 days and doctors struggling to keep up experience burnout at peak rates. Successful generative AI applications include data analytics, document generation, and question answering on medical databases, however, generative AI is yet to make its way into hospitals. OBJECTIVE We aim to accelerate this transition by offering Rhazes, an AI-Assistant for doctors to assist them with paperwork and analytical tasks in medicine. METHODS Storage, hosting, AI services and LLMs are provided by Microsoft Azure and OpenAI. Retrieval augmented generation is used to for diagnosis and billing and WebRTC protocol is used for telemedicine. RESULTS We made Rhazes available on the web free of charge and it is ready to help physicians with notetaking, clinical decisions and patient management starting from the initial consultation all the way to patient discharge. CONCLUSIONS In the future we aim to pilot Rhazes in hospitals and conduct a thorough evaluation of its performance. INTERNATIONAL REGISTERED REPORT RR2-https://doi.org/10.1002/hcs2.79

  • Open Access

    Mechanism design for a fair and equitable approach to global vaccine distribution: The case of COVID-19

    by Khaled Abedrabboh, Lolwa Al-Majid, Zaid Al-Fagih, Luluwah Al-Fagih

    Vaccines are one of the most effective tools humanity has in the fight against pandemics. One of the major challenges of vaccine distribution is achieving fair and equitable allocation across the countries of the world, regardless of their economic wealth. The self-interested behaviour of high-income countries and the underutilisation of vaccines allocated to underprepared countries are some of the failures reported during COVID-19 vaccine roll-out. These shortcomings have motivated the need for a central market mechanism that takes into account the countries’ vulnerability to COVID-19 and their readiness to distribute and administer their allocated vaccines. In this paper, we leverage game theory to study the problem of equitable global vaccine distribution and propose a fair market mechanism that aligns self-interested behaviour with optimal global objectives. First, we model the interaction between a central vaccine provider (e.g. COVAX) and a country reporting its demand as a two-player game, and discuss the Nash and mixed Nash equilibria of that game. Then, we propose a repeated auction mechanism with an artificial payment system for allocating vaccines among participating countries, where each auction round is based on a Vickrey-Clarke-Groves (VCG) mechanism. The proposed allocation mechanism aims at minimising deaths and incentivises the self-interested countries to report their demand truthfully. Compared with real-world COVAX allocation decisions, our results show that the proposed auction mechanism achieves more efficient outcomes that maximise the number of averted deaths. Pragmatic considerations are investigated and policy recommendations are discussed.