Artificial Intelligence vs Human Clinicians in Esophagogastroduodenoscopy Appropriateness: A Comparative Study Using Clinical Vignettes
- 19 November 2025
- journal article
- Published by Wolters Kluwer Health in American Journal of Gastroenterology
- Vol. 121 (4) , 1025-1030
Abstract
INTRODUCTION: Large language models are increasingly used in clinical decision-making, but their role in appropriateness-based indications is uncertain. METHODS: We conducted a vignette-based Italian survey comparing esophagogastroduodenoscopy appropriateness by 5 artificial intelligence (AI) (ChatGPT-4.0, ChatGPT-4.5, Gemini, Claude AI, OpenEvidence) at 2 times (April and September 2025) with gastroenterologists, residents, and general practitioners, using American Society for Gastrointestinal Endoscopy and European Society of Gastrointestinal Endoscopy guidelines. RESULTS: A total of 135 physicians participated. AI performance varied over time: accuracies ranged from 50%–90% in April to 63%–80% in September, with ChatGPT-4.5 and ChatGPT-4.0 outperforming physicians in September. DISCUSSION: Temporal and prompt variability highlight the need for multirun, longitudinal evaluation before clinical adoption.Keywords
This publication has 15 references indexed in Scilit:
- Evaluation of large language models as a diagnostic tool for medical learners and clinicians using advanced prompting techniquesPLOS ONE, 2025
- Management of epithelial precancerous conditions and early neoplasia of the stomach (MAPS III): European Society of Gastrointestinal Endoscopy (ESGE), European Helicobacter and Microbiota Study Group (EHMSG) and European Society of Pathology (ESP) Guideline update 2025Endoscopy, 2025
- Large Language Models Applied to Health Care Tasks May Improve Clinical Efficiency, Value of Care Rendered, Research, and Medical EducationArthroscopy, 2024
- Evaluation and mitigation of the limitations of large language models in clinical decision-makingNature Medicine, 2024
- Risk Perception, Acceptance, and Trust of Using AI in Gastroenterology Practice in the Asia-Pacific Region: Web-Based Survey StudyJMIR AI, 2024
- Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability StudyJournal of Medical Internet Research, 2023
- Large language models in medicineNature Medicine, 2023
- Artificial Intelligence and Primary Care Research: A Scoping ReviewThe Annals of Family Medicine, 2020
- Application of Artificial Intelligence to Gastroenterology and HepatologyGastroenterology, 2020
- Appropriate use of GI endoscopyGastrointestinal Endoscopy, 2012