Evaluating Guideline Adherence in Gemini‐Powered Dental Trauma Workflows: Standalone Gemini Chat vs. Document‐Grounded NotebookLM
Authors:
Top Cited Papers
Open Access
- 2 March 2026
- journal article
- Published by Wiley in Dental Traumatology
- Vol. 42 (5) , 659-667
Abstract
Aim: The aim of this study was to compare the accuracy and inter‐account consistency of two Google Gemini–powered, user‐facing workflows for dental trauma decision support: standalone Gemini chat and NotebookLM, a document‐grounded workflow that generates responses grounded in uploaded European Society of Endodontology and International Association of Dental Traumatology guideline documents, when answering dichotomous (yes/no) clinical questions on the management of traumatized permanent teeth. Methodology: A cross‐sectional simulation was conducted using 99 dichotomous (yes/no) questions derived from the European Society of Endodontology and International Association of Dental Traumatology guidelines. Three academic endodontists submitted each question to Gemini and NotebookLM using three independent Google accounts, generating 297 responses per workflow. Accuracy was defined as exact agreement with guideline‐based answers, and consistency as the proportion of identical responses across the three trials. Statistical analyses included Wald and Wilson 95% confidence intervals, Fleiss' kappa for inter‐account agreement, and Pearson's chi‐squared tests to compare proportions. Results: Gemini demonstrated an overall accuracy of 83.83% (95% CI: 75.08–90.47) and a consistency of 74.74% (κ = 0.84). NotebookLM showed higher accuracy (92.93%; 95% CI: 85.97–97.11) and perfect consistency (100%; κ = 1.00). While the difference in accuracy did not reach statistical significance (p = 0.076), NotebookLM exhibited significantly greater consistency (p < 0.001). Conclusions: The responses generated from the guidelines were highly consistent with both workflows. Document grounding may enhance repeatability and alignment with guideline‐derived decision points for structured dichotomous inquiries, as evidenced by NotebookLM's ability to achieve complete inter‐account consistency and to quantitatively increase accuracy. These results are the outcome of workflow‐level benchmarking; therefore, clinical utility cannot be inferred solely from them; professional oversight and additional validation remain necessary before any clinical application.Keywords
Funding Information
- Agencia Nacional de Investigación y Desarrollo (72250040)
This publication has 21 references indexed in Scilit:
- Large language models in methodological quality evaluation of radiomics research based on METRICS: ChatGPT vs NotebookLM vs radiologistEuropean Journal of Radiology, 2025
- RETRACTED: Poe or Gemini for fostering writing skills in Japanese upper‐intermediate learners: Uncovering the consequences on positive emotions, boredom to write, academic self‐efficacy and writing developmentBritish Educational Research Journal, 2025
- Application of large language models in disease diagnosis and treatmentChinese Medical Journal, 2024
- Performance of Artificial Intelligence Chatbots in Responding to Patient Queries Related to Traumatic Dental Injuries: A Comparative StudyDental Traumatology, 2024
- Evidence‐Based Potential of Generative Artificial Intelligence Large Language Models on Dental Avulsion: ChatGPT Versus GeminiDental Traumatology, 2024
- Pedagogy and generative artificial intelligence: Applying the PICRAT model to Google NotebookLMMedical Teacher, 2024
- Accuracy and Consistency of Gemini Responses Regarding the Management of Traumatized Permanent TeethDental Traumatology, 2024
- Evaluation of validity and reliability of AI Chatbots as public sources of information on dental traumaDental Traumatology, 2024
- Large language models illuminate a progressive pathway to artificial intelligent healthcare assistantMedicine Plus, 2024
- Assessment of artificial intelligence applications in responding to dental traumaDental Traumatology, 2024