Assessing the ability of large language models to summarize and generate maxillofacial prosthetic treatment options
Open Access
- 14 April 2026
- journal article
- Published by Wiley in Journal of Prosthodontics
Abstract
Purpose: The aim was to evaluate the ability of four large language models (LLMs) (OpenAI's ChatGPT‐3.5, Microsoft 365 Copilot, DeepSeek‐R1, and Google Gemini 2.5 Pro) to develop treatment options when presented with clinical cases published in the maxillofacial prosthodontics literature. Materials and Methods: Six maxillofacial case reports were fed to the LLMs following a prompt that requested prosthodontic treatment options from the perspective of a prosthodontist. Expert evaluators scored the relevance, clarity, depth, focus, and coherence of the responses. Statistical analyses, including descriptive statistics, two‐way analysis of variance (ANOVA), post hoc Tukey tests, Pearson correlation analyses, and intraclass correlation coefficients (ICCs), were performed ( α < 0.05). Results: There were significant differences among the total mean relevance ( p = 0.003), clarity ( p = 0.006), depth ( p < 0.001), focus ( p < 0.001), and coherence ( p < 0.001) scores of chatbots. Copilot consistently scored the lowest, and Gemini or DeepSeek scored the highest for all five factors. Depth ( p = 0.006), focus ( p = 0.024), and coherence ( p = 0.013) scores of senior prosthodontists were slightly higher than those of junior prosthodontists. Pearson correlation analysis revealed positive correlations between the total mean scores for all five factors ( p < 0.001). Conclusions: The study demonstrates the ability of LLMs to develop maxillofacial prosthetic treatment plans tailored to specific clinical scenarios. There were significant differences between the abilities of the LLMs evaluated in this study. Copilot scored the lowest for all factors evaluated, and Gemini and/or DeepSeek scored the highest.Keywords
This publication has 44 references indexed in Scilit:
- Performance comparison of large language models in treatment planning for the restoration of endodontically treated teeth over timeJournal of Dentistry, 2025
- Assessing the performance of Microsoft Copilot, GPT-4 and Google Gemini in ophthalmologyCanadian Journal of Ophthalmology, 2025
- Artificial intelligence-large language models (AI-LLMs) for reliable and accurate cardiotocography (CTG) interpretation in obstetric practiceComputational and Structural Biotechnology Journal, 2025
- Assessment of Large Language Models (LLMs) in decision-making support for gynecologic oncologyComputational and Structural Biotechnology Journal, 2024
- Performance of Multimodal Artificial Intelligence Chatbots Evaluated on Clinical Oncology CasesJAMA Network Open, 2024
- Accuracy and consistency of chatbots versus clinicians for answering pediatric dentistry questions: A pilot studyJournal of Dentistry, 2024
- Natural Language Processing: Chances and Challenges in DentistryJournal of Dentistry, 2023
- Ethical Concerns Regarding the Use of Large Language Models in HealthcareEJVES Vascular Forum, 2023
- A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability ResearchJournal of Chiropractic Medicine, 2016
- Mandibular Reconstruction: OverviewJournal of Maxillofacial and Oral Surgery, 2015