The role of generative language systems in increasing patient awareness of colon cancer screening
Authors:
- Marcello MaidaMMMarcello Maida
- Yuichi MoriYMYuichi Mori
- Antonio FacciorussoAntonio Facciorusso
- the AI-CORE (Artificial Intelligence COlorectal cancer Research) Working GroupTGthe AI-CORE (Artificial Intelligence COlorectal cancer Research) Working Group
- 14 August 2024
- journal article
- research article
- Published by Georg Thieme Verlag KG in Endoscopy
- Vol. 57 (03) , 262-268
Abstract
Background This study aimed to evaluate the effectiveness of ChatGPT (Chat Generative Pretrained Transformer) in answering patientsʼ questions about colorectal cancer (CRC) screening, with the ultimate goal of enhancing patients' awareness and adherence to national screening programs. Methods 15 questions on CRC screening were posed to ChatGPT4. The answers were rated by 20 gastroenterology experts and 20 nonexperts in three domains (accuracy, completeness, and comprehensibility), and by 100 patients in three dichotomic domains (completeness, comprehensibility, and trustability). Results According to expert rating, the mean (SD) accuracy score was 4.8 (1.1), on a scale ranging from 1 to 6. The mean (SD) scores for completeness and comprehensibility were 2.1 (0.7) and 2.8 (0.4), respectively, on scales ranging from 1 to 3. Overall, the mean (SD) accuracy (4.8 [1.1] vs. 5.6 [0.7]; P < 0.001) and completeness scores (2.1 [0.7] vs. 2.7 [0.4]; P < 0.001) were significantly lower for the experts than for the nonexperts, while comprehensibility was comparable among the two groups (2.8 [0.4] vs. 2.8 [0.3]; P = 0.55). Patients rated all questions as complete, comprehensible, and trustable in between 97 % and 100 % of cases. Conclusions ChatGPT shows good performance, with the potential to enhance awareness about CRC and improve screening outcomes. Generative language systems may be further improved after proper training in accordance with scientific evidence and current guidelines.Keywords
This publication has 14 references indexed in Scilit:
- Accuracy, Reliability, and Comprehensibility of ChatGPT-Generated Medical Responses for Patients With Nonalcoholic Fatty Liver DiseaseClinical Gastroenterology and Hepatology, 2023
- ChatGPT Answers Common Patient Questions About ColonoscopyGastroenterology, 2023
- Effect of Colonoscopy Screening on Risks of Colorectal Cancer and Related DeathNew England Journal of Medicine, 2022
- Effect of Patient Portal Messaging Before Mailing Fecal Immunochemical Test Kit on Colorectal Cancer Screening RatesJAMA Network Open, 2022
- A Patient Portal-Based Commitment Device to Improve Adherence with Screening for Colorectal Cancer: a Retrospective Observational StudyJournal of General Internal Medicine, 2021
- Promoting colorectal cancer screening: a systematic review and meta-analysis of randomised controlled trials of interventions to increase uptakeHealth Psychology Review, 2020
- The Global Burden of Cancer 2013JAMA Oncology, 2015
- Long-Term Colorectal-Cancer Mortality after Adenoma RemovalNew England Journal of Medicine, 2014
- Attendance and Yield Over Three Rounds of Population-Based Fecal Immunochemical Test ScreeningAmerican Journal of Gastroenterology, 2014
- Coefficient Alpha and the Internal Structure of TestsPsychometrika, 1951