Performance of ChatGPT and Dental Students on Concepts of Periodontal Surgery
Authors:
- 24 October 2024
- journal article
- research article
- Published by Wiley in European Journal of Dental Education
- Vol. 29 (1) , 36-43
Abstract
Introduction: As a large language model, chat generative pretrained transformer (ChatGPT) has provided a valuable tool for various medical scenarios with its interactive dialogue‐based interface. However, there is a lack of studies on ChatGPT's effectiveness in handling dental tasks. This study aimed to compare the knowledge and comprehension abilities of ChatGPT‐3.5/4 with that of dental students about periodontal surgery.Materials and Methods: A total of 134 dental students participated in this study. We designed a questionnaire consisting of four questions about the inclination for ChatGPT, 25 multiple‐choice, and one open‐ended question. As the comparison of ChatGPT‐3.5 and 4, the question about the inclination was removed, and the rest was the same. The response time of ChatGPT‐3.5 and 4 as well as the comparison of ChatGPT‐3.5 and 4′ performances with dental students were measured. Regarding students' feedback on the open‐ended question, we also compared the outcomes of ChatGPT‐4′ and teacher's review.Results: On average, ChatGPT‐3.5 and 4 required 3.63 ± 1.18 s (95% confidence interval [CI], 3.14, 4.11) and 12.49 ± 7.29 s (95% CI, 9.48, 15.50), respectively (p < 0.001) for each multiple‐choice question. For these 25 questions, the accuracy was 21.51 ± 2.72, 14 and 20 for students, ChatGPT‐3.5 and 4, respectively. Furthermore, the outcomes of ChatGPT‐4's review were consistent with that of teacher's review.Conclusions: For dental examinations related to periodontal surgery, ChatGPT's accuracy was not yet comparable to that of the students. Nevertheless, ChatGPT shows promise in assisting students with the curriculum and helping practitioners with clinical letters and reviews of students' textual descriptions.Keywords
This publication has 39 references indexed in Scilit:
- Artificial intelligence: ChatGPT as a disruptive didactic strategy in dental educationJournal of Dental Education, 2024
- Evaluating ChatGPT in Medical Contexts: The Imperative to Guard Against Hallucinations and Partial AccuraciesClinical Gastroenterology and Hepatology, 2023
- ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reportsEuropean Radiology, 2023
- ChatGPT and Generating a Differential Diagnosis Early in an Emergency Department PresentationAnnals of Emergency Medicine, 2023
- Performance and exploration of ChatGPT in medical examination, records and education in Chinese: Pave the way for medical AIInternational Journal of Medical Informatics, 2023
- Role of ChatGPT-4 for Medical ResearchersAnnals of Biomedical Engineering, 2023
- Success of ChatGPT, an AI language model, in taking the French language version of the European Board of Ophthalmology examination: A novel approach to medical knowledge assessmentJournal Français d'Ophtalmologie, 2023
- Use of ChatGPT on Taiwan’s Examination for Medical DoctorsAnnals of Biomedical Engineering, 2023
- Harnessing the Potential of ChatGPT in Breast Reconstruction: A Revolution in Patient Communication and EducationAesthetic Plastic Surgery, 2023
- ChatGPT and large language model (LLM) chatbots: The current state of acceptability and a proposal for guidelines on utilization in academic medicineJournal of Pediatric Urology, 2023