Performance of generative artificial intelligence in oral and maxillofacial radiology based on the board-certification examination of Japan
Authors:
Open Access
- 20 August 2026
- journal article
- research article
- Published by Springer Nature in Oral Radiology
- Vol. 42 (4) , 1184-1191
Abstract
Objective: To evaluate the performance and potential utility of generative artificial intelligence (AI) in oral and maxillofacial radiology using the board-certification examination administered by the Japanese Society for Oral and Maxillofacial Radiology (JSOMR). Methods: The responses generated by ChatGPT for multiple-choice questions from the board-certification examination of the JSOMR over the three-year period from 2020 to 2022 were assessed. The questions were manually entered individually as prompts for GPT-3.5, GPT-4, and GPT-5, which are the models available from ChatGPT. The accuracy was calculated according to examination year, question format, and level of taxonomy. Results: GPT-3.5 achieved an accuracy of 40.3% for the three years (42.9%, 42.0%, and 36.0% for 2020, 2021, and 2022, respectively), that of GPT-4 was 67.8% (67.3%, 74.0%, and 62.0%, respectively), and that of GPT-5 was 76.5% (79.6%, 78.0%, and 72.0%, respectively). GPT-5’s results exceeded the passing score for each year with the accuracy significantly outperformed that of GPT-3.5 and GPT-4. Regarding performance according to the question format, GPT-5 performed significantly superior to the earlier models, especially on two-answer questions. Conclusions: On the board-certification examination of the JSOMR, the performance of GPT-5 was significantly superior to that of GPT-3.5 and GPT-4. This suggests that, given the rapid development of generative AI, GPT-5 has reached a level of text-based knowledge equivalent to that assessed in the board-certification examination of the JSOMR.Keywords
Funding Information
- Okayama University
This publication has 22 references indexed in Scilit:
- The performance of large language models in dentomaxillofacial radiology: a systematic reviewDentomaxillofacial Radiology, 2025
- Performance Assessment of ChatGPT for the Board Qualification Examination of the Japanese Society for Oral and Maxillofacial RadiologyTechnology, Knowledge and Learning, 2025
- How well do large language model-based chatbots perform in oral and maxillofacial radiology?Dentomaxillofacial Radiology, 2024
- Preliminary Study of Dental Caries Detection by Deep Neural Network Applying Domain-Specific Transfer LearningJournal of Medical and Biological Engineering, 2024
- Evaluating GPT-4V’s performance in the Japanese national dental examination: A challenge exploredJournal of Dental Sciences, 2023
- Performance evaluation of ChatGPT, GPT-4, and Bard on the official board examination of the Japan Radiology SocietyJapanese Journal of Radiology, 2023
- Large language models in medicineNature Medicine, 2023
- Natural language processing: state of the art, current trends and challengesMultimedia Tools and Applications, 2022
- Natural Language Processing: from Bedside to EverywhereYearbook of Medical Informatics, 2022
- Artificial intelligence in radiologyNature Reviews Cancer, 2018