Exploring the role of artificial intelligence, large language models: Comparing patient‐focused information and clinical decision support capabilities to the gynecologic oncology guidelines
Authors:
Open Access
- 20 August 2024
- journal article
- research article
- Published by Wiley in International Journal of Gynecology & Obstetrics
- Vol. 168 (2) , 419-427
Abstract
Gynecologic cancer requires personalized care to improve outcomes. Large language models (LLMs) hold the potential to provide intelligent question‐answering with reliable information about medical queries in clear and plain English, which can be understood by both healthcare providers and patients. We aimed to evaluate two freely available LLMs (ChatGPT and Google's Bard) in answering questions regarding the management of gynecologic cancer. The LLMs' performances were evaluated by developing a set questions that addressed common gynecologic oncologic findings from a patient's perspective and more complex questions to elicit recommendations from a clinician's perspective. Each question was presented to the LLM interface, and the responses generated by the artificial intelligence (AI) model were recorded. The responses were assessed based on the adherence to the National Comprehensive Cancer Network and European Society of Gynecological Oncology guidelines. This evaluation aimed to determine the accuracy and appropriateness of the information provided by LLMs. We showed that the models provided largely appropriate responses to questions regarding common cervical cancer screening tests and BRCA ‐related questions. Less useful answers were received to complex and controversial gynecologic oncology cases, as assessed by reviewing the common guidelines. ChatGPT and Bard lacked knowledge of regional guideline variations, However, it provided practical and multifaceted advice to patients and caregivers regarding the next steps of management and follow up. We conclude that LLMs may have a role as an adjunct informational tool to improve outcomes.Keywords
This publication has 16 references indexed in Scilit:
- Let's chat about cervical cancer: Assessing the accuracy of ChatGPT responses to cervical cancer questionsGynecologic Oncology, 2023
- Large language models in medicineNature Medicine, 2023
- Large language model (ChatGPT) as a support tool for breast tumor boardnpj Breast Cancer, 2023
- ESGO/ESTRO/ESP Guidelines for the management of patients with cervical cancer – Update 2023*International Journal of Gynecological Cancer, 2023
- Appropriateness of Breast Cancer Prevention and Screening Recommendations Provided by ChatGPTRadiology, 2023
- ChatGPT and the Future of Medical WritingRadiology, 2023
- Appropriateness of Cardiovascular Disease Prevention Recommendations Obtained From a Popular Online Chat-Based Artificial Intelligence ModelJAMA, 2023
- FUTURE OF THE LANGUAGE MODELS IN HEALTHCARE: THE ROLE OF CHATGPTABCD. Arquivos Brasileiros de Cirurgia Digestiva (São Paulo), 2023
- European Society of Gynaecological Oncology guidelines for the peri-operative management of advanced ovarian cancer patients undergoing debulking surgeryInternational Journal of Gynecological Cancer, 2021
- ESGO/ESTRO/ESP guidelines for the management of patients with endometrial carcinomaInternational Journal of Gynecological Cancer, 2021