Comparison of Human and Artificial Intelligence ( AI ) in Writing and Rating Restorative Dentistry Essays
- 25 September 2025
- journal article
- Published by Wiley in European Journal of Dental Education
- Vol. 30 (3) , 1026-1037
Abstract
Aim: This study compared the writing quality of AI‐ and student‐authored essays, evaluated faculty members' reliability to differentiate between essays authored by AI and students, and assessed the scoring accuracy between human and AI raters using a standardised rubric. Methods: Four topics were selected from a preclinical operative and aesthetic dentistry course. Each topic was presented as four essays authored by two students and two AI tools (ChatGPT4 and Gemini) ( N = 48). Then, the 16 essays were evaluated either by three blinded experts and two AI tool raters (ChatGPT4 and Gemini) using a modified Universal Science Writing Rubric. The Shapiro–Wilk W test assessed data normality. Kruskal‐Wallis, Dunn's Pairwise, Wilcoxon Signed‐rank, and Friedman tests analysed the writing performances and inter‐rater reliabilities with a significance level of ( α = 0.05). Results: Significant differences were found in evaluating scientific content ( Z = 9.28, p = 0.005) and interpreting scientific content ( Z = 6.74, p = 0.021) between AI‐ and student‐authored essays. ChatGPT4‐authored essays differed significantly in scientific content from both Gemini‐ and student‐authored essays, with further differences in interpretation between ChatGPT4‐ and student‐authored essays ( p = 0.011). Faculty members correctly identified 75% of essay authors. No significant differences were found between raters using ChatGPT4 or Gemini, while a marginally significant difference was observed between human raters in the overall score, though not in specific parameters. Conclusions: Gemini's scores closely matched those of human‐authored essays, aligning more with human raters than ChatGPT‐4. AI's capacity emulated human writing, though differences are noticeable to trained faculty members. There are evident disparities in content quality and organisation between AI‐ and human‐authored work.Keywords
Funding Information
- Princess Nourah Bint Abdulrahman University ((PNURSP2025R350))
This publication has 16 references indexed in Scilit:
- Artificial Intelligence in Dental Education: Opportunities and Challenges of Large Language Models and Multimodal Foundation ModelsJMIR Medical Education, 2024
- Artificial intelligence-powered dentistry: Probing the potential, challenges, and ethicality of artificial intelligence in dentistryDigital Health, 2024
- Exploring Artificial Intelligence in Academic Essay: Higher Education Student's PerspectiveInternational Journal of Educational Research Open, 2023
- Shaping the Future of Education: Exploring the Potential and Consequences of AI and ChatGPT in Educational SettingsEducation Sciences, 2023
- Ethical use of Artificial Intelligence in Health Professions Education: AMEE Guide No. 158Medical Teacher, 2023
- Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive studyJournal of Educational Evaluation for Health Professions, 2023
- Legal and Ethical Consideration in Artificial Intelligence in Healthcare: Who Takes Responsibility?Frontiers in Surgery, 2022
- Artificial Intelligence in Education: A ReviewIEEE Access, 2020
- A hierarchical classification approach to automated essay scoringAssessing Writing, 2015
- The Measurement of Observer Agreement for Categorical DataBiometrics, 1977