Utility of Generative Artificial Intelligence for Japanese Medical Interview Training: A Randomized Crossover Pilot Study (Preprint)
Open Access
- 12 May 2025
- journal article
- research article
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 11, e77332
Abstract
Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: The medical interview remains a cornerstone of clinical training. There is growing interest in applying generative artificial intelligence (AI) in medical education, including medical interview training. However, its utility in culturally and linguistically specific contexts, including Japanese, remains underexplored. This study investigated the utility of generative AI for Japanese medical interview training. Objective: This pilot study aimed to evaluate the utility of generative AI as a tool for medical interviews training by comparing its performance with that of traditional face-to-face training methods using a simulated patient. Methods: We conducted a randomized crossover pilot study involving 20 postgraduate year 1-2 physicians from a university hospital. Participants were randomly allocated into two groups. Group A began with an AI-based station on a case involving abdominal pain, followed by a traditional station with a standardized patient presenting chest pain. Group B followed the reverse order, starting with the traditional station for abdominal pain, and subsequently within AI-based station for the chest pain scenario. In the AI-based stations, participants interacted with a GPTs-configured platform that simulated patient behaviors. GPTs are customizable versions of ChatGPT adapted for specific purposes. The traditional stations involved face-to-face interviews with a simulated patient. Both groups used identical, standardized case scenarios to ensure uniformity. Two independent evaluators, blinded to the study conditions, assessed participants' performances using six defined metrics: patient care and communication, history taking, physical examination, accuracy and clarity of transcription, clinical reasoning, and patient management. A 6-point Likert scale was employed for scoring. Discrepancy between the evaluators resolved through discussion. To ensure cultural and linguistic authenticity, all interviews and evaluations were conducted in Japanese. Results: AI-based stations scored lower across most categories, particularly in patient care and communication, than traditional stations (4.48 vs. 4.95, P=.009). However, AI-based stations demonstrated comparable performance in clinical reasoning, with a non-significant difference (4.43 vs. 4.85, P=.10). Conclusions: The comparable performance of generative AI in clinical reasoning highlights its potential as a complementary tool in medical interview training. One of its main advantages lies in enabling self-learning, allowing trainees to independently practice interviews without the need for simulated patients. Nonetheless, the lower scores in patient care and communication underline the importance of maintaining traditional methods that capture the nuances of human interaction. These findings support the adoption of hybrid training models that combine generative AI with conventional approaches to enhance the overall effectiveness of medical interview training in Japan. Clinical Trial: UMIN-CTR UMIN000053747; https://center6.umin.ac.jp/cgi-open-bin/ctr_e/ctr_view.cgi?recptno=R000061336.Keywords
This publication has 65 references indexed in Scilit:
- An Overview of AI Platforms, Frameworks, Libraries, and Processors2024
- User engagement, attitudes, and the effectiveness of chatbots as a mental health intervention: A systematic reviewComputers in Human Behavior: Artificial Humans, 2024
- Chatbots applications in education: A systematic reviewComputers and Education: Artificial Intelligence, 2021
- Deep learning with GPUs2021
- Competency in medical history taking—the training physicians’ viewWiener klinische Wochenschrift, 2018
- Randomization in Clinical TrialsJAMA, 2018
- Teaching the Medical Interview: Methods and Key Learning Issues in a Faculty Development CourseJournal of General Internal Medicine, 2007
- Teaching Medical InterviewingArchives of internal medicine (1960), 1992
- Clinical Methods: The History, Physical, and Laboratory ExaminationsJAMA, 1990
- A history of history-takingJournal of General Internal Medicine, 1987