BZBoyang Zhuang

Boyang Zhuang

*This is an algorithmically generated author record based on publicly available information.

Boyang Zhuang has a total of 14 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Top Cited Papers
    Open Access

    Accuracy of Large Language Models When Answering Clinical Research Questions: Systematic Review and Network Meta-Analysis

    by Ling Wang, Jinglin Li, Boyang Zhuang, Shasha Huang, Meilin Fang, Cunze Wang, Wen Li, Mohan Zhang, Shurong Gong

    Background: Large language models (LLMs) have flourished and gradually become an important research and application direction in the medical field. However, due to the high degree of specialization, complexity, and specificity of medicine, which results in extremely high accuracy requirements, controversy remains about whether LLMs can be used in the medical field. More studies have evaluated the performance of various types of LLMs in medicine, but the conclusions are inconsistent. Objective: This study uses a network meta-analysis (NMA) to assess the accuracy of LLMs when answering clinical research questions to provide high-level evidence-based evidence for its future development and application in the medical field. Methods: In this systematic review and NMA, we searched PubMed, Embase, Web of Science, and Scopus from inception until October 14, 2024. Studies on the accuracy of LLMs when answering clinical research questions were included and screened by reading published reports. The systematic review and NMA were conducted to compare the accuracy of different LLMs when answering clinical research questions, including objective questions, open-ended questions, top 1 diagnosis, top 3 diagnosis, top 5 diagnosis, and triage and classification. The NMA was performed using Bayesian frequency theory methods. Indirect intercomparisons between programs were performed using a grading scale. A larger surface under the cumulative ranking curve (SUCRA) value indicates a higher ranking of the corresponding LLM accuracy. Results: The systematic review and NMA examined 168 articles encompassing 35,896 questions and 3063 clinical cases. Of the 168 studies, 40 (23.8%) were considered to have a low risk of bias, 128 (76.2%) had a moderate risk, and none were rated as having a high risk. ChatGPT-4o (SUCRA=0.9207) demonstrated strong performance in terms of accuracy for objective questions, followed by Aeyeconsult (SUCRA=0.9187) and ChatGPT-4 (SUCRA=0.8087). ChatGPT-4 (SUCRA=0.8708) excelled at answering open-ended questions. In terms of accuracy for top 1 diagnosis and top 3 diagnosis of clinical cases, human experts (SUCRA=0.9001 and SUCRA=0.7126, respectively) ranked the highest, while Claude 3 Opus (SUCRA=0.9672) performed well at the top 5 diagnosis. Gemini (SUCRA=0.9649) had the highest rated SUCRA value for accuracy in the area of triage and classification. Conclusions: Our study indicates that ChatGPT-4o has an advantage when answering objective questions. For open-ended questions, ChatGPT-4 may be more credible. Humans are more accurate at the top 1 diagnosis and top 3 diagnosis. Claude 3 Opus performs better at the top 5 diagnosis, while for triage and classification, Gemini is more advantageous. This analysis offers valuable insights for clinicians and medical practitioners, empowering them to effectively leverage LLMs for improved decision-making in learning, diagnosis, and management of various clinical scenarios. Trial Registration: PROSPERO CRD42024558245; https://www.crd.york.ac.uk/PROSPERO/view/CRD42024558245


  • Open Access

    Analysis of Predictors of Adverse Events and Mortality Risk Associated With IL‐6 Inhibitors: A Pharmacovigilance Study Using the FDA Adverse Event Reporting System Database

    by Meilin Fang, Jinglin Li, Boyang Zhuang, Weijie Liang, Ling Wang, Cunze Wang, Wujin Chen, Fangqing Cai, Junshan Ruan, Yishun Jin

    Aim: This study aimed to investigate post‐marketing adverse events (AEs) of interleukin‐6 (IL‐6) inhibitors, and to explore risk factors for death. Method: Disproportionality analyses were conducted on adverse event cases of IL‐6 inhibitors reported to the US Food and Drug Administration Adverse Event Reporting System (FAERS) from the time of drug launch until the fourth quarter of 2023. Univariate and multivariate logistic regression analyses were carried out utilizing patient‐related clinical information, and prediction models for IL‐6 inhibitor‐related mortality risk were developed by incorporating patient age and weight factors. Results: A total of 63,445 reports were retrieved, with the majority of known age groups falling between 18 and 64 years. Most reports were submitted by consumers and physicians, predominantly from the United States. Tocilizumab was associated with AEs such as drug intolerance and infection, while sarilumab showed symptoms of pain and condition aggravated. Siltuximab was linked to disease progression and thrombocytopenia. The median time to AEs with IL‐6 inhibitors was 74 days (interquartile range [IQR] 10‐311), mostly occurring within 1 month. Factors such as age, propionic acid derivative, infections and infestations, nervous system disorders, and immune system disorders were independent risk factors for deaths related to IL‐6 inhibitor use ( p < 0.05). The mortality risk prediction model demonstrated good discriminatory power and clinical applicability in both the training set (AUC 0.6968) and the validation set (AUC 0.7502). Conclusion: Our postmarketing pharmacovigilance analysis revealed the types and incidence of AEs related to IL‐6 inhibitors. Column line diagrams may be useful for clinical assessment of the occurrence of death and have high clinical utility.


  • Open Access

    Accuracy of Large Language Models When Answering Clinical Research Questions: Systematic Review and Network Meta-Analysis (Preprint)

    by Ling Wang, Jinglin Li, Boyang Zhuang, Shasha Huang, Meilin Fang, Cunze Wang, Wen Li, Mohan Zhang, Shurong Gong

    BACKGROUND Large language models (LLMs) have flourished and gradually become an important research and application direction in the medical field. However, due to the high degree of specialization, complexity, and specificity of medicine, which results in extremely high accuracy requirements, controversy remains about whether LLMs can be used in the medical field. More studies have evaluated the performance of various types of LLMs in medicine, but the conclusions are inconsistent. OBJECTIVE This study uses a network meta-analysis (NMA) to assess the accuracy of LLMs when answering clinical research questions to provide high-level evidence-based evidence for its future development and application in the medical field. METHODS In this systematic review and NMA, we searched PubMed, Embase, Web of Science, and Scopus from inception until October 14, 2024. Studies on the accuracy of LLMs when answering clinical research questions were included and screened by reading published reports. The systematic review and NMA were conducted to compare the accuracy of different LLMs when answering clinical research questions, including objective questions, open-ended questions, top 1 diagnosis, top 3 diagnosis, top 5 diagnosis, and triage and classification. The NMA was performed using Bayesian frequency theory methods. Indirect intercomparisons between programs were performed using a grading scale. A larger surface under the cumulative ranking curve (SUCRA) value indicates a higher ranking of the corresponding LLM accuracy. RESULTS The systematic review and NMA examined 168 articles encompassing 35,896 questions and 3063 clinical cases. Of the 168 studies, 40 (23.8%) were considered to have a low risk of bias, 128 (76.2%) had a moderate risk, and none were rated as having a high risk. ChatGPT-4o (SUCRA=0.9207) demonstrated strong performance in terms of accuracy for objective questions, followed by Aeyeconsult (SUCRA=0.9187) and ChatGPT-4 (SUCRA=0.8087). ChatGPT-4 (SUCRA=0.8708) excelled at answering open-ended questions. In terms of accuracy for top 1 diagnosis and top 3 diagnosis of clinical cases, human experts (SUCRA=0.9001 and SUCRA=0.7126, respectively) ranked the highest, while Claude 3 Opus (SUCRA=0.9672) performed well at the top 5 diagnosis. Gemini (SUCRA=0.9649) had the highest rated SUCRA value for accuracy in the area of triage and classification. CONCLUSIONS Our study indicates that ChatGPT-4o has an advantage when answering objective questions. For open-ended questions, ChatGPT-4 may be more credible. Humans are more accurate at the top 1 diagnosis and top 3 diagnosis. Claude 3 Opus performs better at the top 5 diagnosis, while for triage and classification, Gemini is more advantageous. This analysis offers valuable insights for clinicians and medical practitioners, empowering them to effectively leverage LLMs for improved decision-making in learning, diagnosis, and management of various clinical scenarios. CLINICALTRIAL PROSPERO CRD42024558245; https://www.crd.york.ac.uk/PROSPERO/view/CRD42024558245