Gut Instincts, Machine Decisions: Evaluating AI Accuracy in the Diagnosis and Treatment of Disorders of Gut‐Brain Interaction
Open Access
- 10 July 2026
- journal article
- research article
- Published by Wiley in Neurogastroenterology & Motility
- Vol. 38 (7) , e70393
Abstract
Background: Disorders of Gut‐Brain Interaction (DGBIs) are common yet challenging diagnoses for gastroenterologists and primary care clinicians. Large language models (LLMs) may support clinical management, but their accuracy remains poorly defined. This study evaluated the diagnostic and treatment accuracy of five LLMs using a convenience sample of clinical scenarios from the Rome IV Multidimensional Clinical Profile (MDCP). Methods: Sixty eight cases representing the DGBIs were entered into commonly accessible, untrained LLMs: ChatGPT 4.0, Google Gemini 2.5 Pro, Microsoft Copilot, OpenEvidence, and Perplexity. A standardized prompt elicited a diagnosis, treatment options, and confidence ratings for each diagnosis. Outputs were assessed against the MDCP recommendations and expert opinion. Key Results: Diagnostic accuracy across models demonstrated Perplexity at 74%, ChatGPT and Google Gemini at 72% respectively, and Microsoft Copilot and OpenEvidence at 65% respectively. Treatment accuracy across models demonstrated ChatGPT, Google Gemini, and Microsoft Copilot generating accurate treatment options between 53% and 54% of cases each, compared with 41% for OpenEvidence and Perplexity each. No statistically significant differences were observed between AI models for diagnostic or treatment accuracy. Confidence ratings were uniformly high regardless of accuracy, with the mean scores ranging from 93% to 96% and standard deviation ranging from 1% to 4%. Conclusions and Inferences: Untrained LLMs demonstrated promising but imperfect diagnostic and treatment performance in this study of a convenience sample of DGBI cases. High confidence despite diagnostic errors and inconsistent treatment recommendations highlights the need for cautious clinical integration and validation.Keywords
This publication has 45 references indexed in Scilit:
- Misdiagnosis of Gastroparesis is Common: A Retrospective Review of Patients Referred to a Tertiary Gastroenterology PracticeClinical Gastroenterology and Hepatology, 2023
- Artificial Intelligence and Anorectal Manometry: Automatic Detection and Differentiation of Anorectal Motility Patterns—A Proof-of-Concept StudyClinical and Translational Gastroenterology, 2022
- The Next EPISOD: Trends in Utilization of Endoscopic Sphincterotomy for Sphincter of Oddi Dysfunction from 2010-2019Clinical Gastroenterology and Hepatology, 2022
- ACG Clinical Guidelines: Management of Benign Anorectal DisordersAmerican Journal of Gastroenterology, 2021
- Deconstructing stigma as a barrier to treating DGBI: Lessons for cliniciansNeurogastroenterology & Motility, 2021
- Worldwide Prevalence and Burden of Functional Gastrointestinal Disorders, Results of Rome Foundation Global StudyGastroenterology, 2021
- Functional gastrointestinal disorders and gut-brain axis: What does the future hold?World Journal of Gastroenterology, 2019
- Association between pelvic floor disorder symptoms and QoL scores with depressive symptoms among pelvic organ prolapse patientsAustralian and New Zealand Journal of Obstetrics and Gynaecology, 2016
- Functional Gastrointestinal Disorders: History, Pathophysiology, Clinical Features, and Rome IVGastroenterology, 2016
- An exploration of the barriers to the confident diagnosis of irritable bowel syndrome: A survey among general practitioners, gastroenterologists and experts in five European countriesUnited European Gastroenterology Journal, 2015