Accuracy and Temporal Consistency of ChatGPT and Gemini in Responding to Textbook and Patient‐Oriented Dental Bleaching Questions: A Multi‐Session Comparative Study
Open Access
- 26 April 2026
- journal article
- Published by Wiley in Journal of Esthetic and Restorative Dentistry
- Vol. 38 (8) , 1553-1563
Abstract
Objective: This study compared the accuracy and temporal consistency of ChatGPT and Gemini in responding to dental bleaching questions across three weekly sessions. Materials and Methods: A total of 280 true/false questions were developed comprising 200 textbook‐based and 80 patient‐oriented frequently asked questions. Both chatbots were queried weekly under controlled conditions. Accuracy was compared using generalized estimating equations, consistency was assessed using Fleiss' kappa, and weekly stability was evaluated using Cochran's Q test. Open‐ended responses were scored for quality and misinformation by two evaluators. Results: For textbook questions, ChatGPT achieved significantly higher accuracy than Gemini (77.7% versus 70.5%, p = 0.0009). For frequently asked questions, both chatbots performed comparably (92.9% versus 90.8%, p = 0.252). Temporal consistency was only fair for textbook questions but almost perfect for frequently asked questions in both chatbots. Both chatbots showed significant upward trends in textbook accuracy across sessions. Gemini received higher global quality scores for open‐ended responses, while misinformation rates were similarly low. Conclusions: Within the limitations of this study, ChatGPT achieved significantly higher accuracy than Gemini for textbook‐based dental bleaching questions, while both chatbots performed comparably for patient‐oriented questions. Temporal consistency differed markedly, with almost perfect consistency for patient‐oriented questions and only fair consistency for textbook‐based questions. Clinical Significance: Chatbot responses to common patient questions about dental bleaching are generally accurate and consistent, but their reliability drops substantially for specialized academic content, suggesting these tools should complement rather than replace professional clinical judgment.Keywords
This publication has 22 references indexed in Scilit:
- Quantifying the speed-accuracy trade-off of large language models on oral and maxillofacial surgery multiple-choice questionsScientific Reports, 2025
- Comparing the performance of ChatGPT, Gemini, and Claude in English and Polish on medical examinationsScientific Reports, 2025
- Comparative Performance of Chatbots in Endodontic Clinical Decision Support: A 4-Day Accuracy and Consistency StudyInternational Dental Journal, 2025
- Information from digital and human sources: A comparison of chatbot and clinician responses to orthodontic questionsAmerican Journal of Orthodontics and Dentofacial Orthopedics, 2025
- Evaluation of the performance of ChatGPT‐4 and ChatGPT‐4o as a learning tool in endodonticsInternational Endodontic Journal, 2025
- Evidence‐Based Potential of Generative Artificial Intelligence Large Language Models on Dental Avulsion: ChatGPT Versus GeminiDental Traumatology, 2024
- Performance of large language artificial intelligence models on solving restorative dentistry and endodontics student assessmentsClinical Oral Investigations, 2024
- Artificial Intelligence in Periodontology: Performance Evaluation of ChatGPT, Claude, and Gemini on the In-service ExaminationmedRxiv, 2024
- Unveiling the ChatGPT phenomenon: Evaluating the consistency and accuracy of endodontic question answersInternational Endodontic Journal, 2023
- Tooth Whitening2023