Integrating AI into clinical education: evaluating general practice trainees’ proficiency in distinguishing AI-generated hallucinations and impacting factors
Top Cited Papers
Open Access
- 19 March 2025
- journal article
- research article
- Published by Springer Nature in BMC Medical Education
- Vol. 25 (1) , 1-9
Abstract
To assess the ability of General Practice (GP) Trainees to detect AI-generated hallucinations in simulated clinical practice, ChatGPT-4o was utilized. The hallucinations were categorized into three types based on the accuracy of the answers and explanations: (1) correct answers with incorrect or flawed explanations, (2) incorrect answers with explanations that contradict factual evidence, and (3) incorrect answers with correct explanations. This multi-center, cross-sectional survey study involved 142 GP Trainees, all of whom were undergoing General Practice Specialist Training and volunteered to participate. The study evaluated the accuracy and consistency of ChatGPT-4o, as well as the Trainees’ response time, accuracy, sensitivity (d’), and response tendencies (β). Binary regression analysis was used to explore factors affecting the Trainees’ ability to identify errors generated by ChatGPT-4o. A total of 137 participants were included, with a mean age of 25.93 years. Half of the participants were unfamiliar with AI, and 35.0% had never used it. ChatGPT-4o’s overall accuracy was 80.8%, which slightly decreased to 80.1% after human verification. However, the accuracy for professional practice (Subject 4) was only 57.0%, and after human verification, it dropped further to 44.2%. A total of 87 AI-generated hallucinations were identified, primarily occurring at the application and evaluation levels. The mean accuracy of detecting these hallucinations was 55.0%, and the mean sensitivity (d’) was 0.39. Regression analysis revealed that shorter response times (OR = 0.92, P = 0.02), higher self-assessed AI understanding (OR = 0.16, P = 0.04), and more frequent AI use (OR = 10.43, P = 0.01) were associated with stricter error detection criteria. The study concluded that GP trainees faced challenges in identifying ChatGPT-4o’s errors, particularly in clinical scenarios. This highlights the importance of improving AI literacy and critical thinking skills to ensure effective integration of AI into medical education.Keywords
All Related Versions
Funding Information
- Wuxi Association for Science and Technology (KX-24-C154)
- Emerging Discipline Leader Program (2024-YZ-HBDTR-ZYY-2024)
- Top Talent Support Program for young and middle-aged people of Wuxi Health Committee (BJRC-8 and HB2023015)
- Taihu Light Basic Research Project, Wuxi Municipal Science and Technology Bureau (K20221023)
This publication has 45 references indexed in Scilit:
- Student interaction with ChatGPT can promote complex critical thinking skillsLearning and Instruction, 2024
- Performance of ChatGPT on American Board of Surgery In-Training Examination Preparation QuestionsJournal of Surgical Research, 2024
- Learning with AI Language Models: Guidelines for the Development and Scoring of Medical Questions for Higher EducationJournal of Medical Systems, 2024
- Three ways ChatGPT helps me in my academic writingNature, 2024
- Primary Care Physicians’ Satisfaction With Interoperable Health Information TechnologyJAMA Network Open, 2024
- Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and FormatJAMA Network Open, 2024
- ChatGPT and medicine: how AI language models are shaping the future and health related careersNature Biotechnology, 2023
- Large language models in medicineNature Medicine, 2023
- Hallucinations in ChatGPT: A Cautionary Tale for Biomedical ResearchersThe American Journal of Medicine, 2023
- Assessing Clinical Reasoning: Targeting the Higher Levels of the PyramidJournal of General Internal Medicine, 2019