Comparing the Performance of ChatGPT-4 and Medical Students on MCQs at Varied Levels of Bloom’s Taxonomy
Authors:
- Ambadasu BharathaABAmbadasu Bharatha
- Nkemcho OjehNONkemcho Ojeh
- Ahbab Mohammad Fazle RabbiARAhbab Mohammad Fazle Rabbi
- Michael H CampbellMCMichael H Campbell
- Kandamaran KrishnamurthyKKKandamaran Krishnamurthy
- Rhaheem Layne-YardeRLRhaheem Layne-Yarde
- Alok KumarAKAlok Kumar
- Kenneth L ConnellKCKenneth L Connell
- Anwarul Azim MajumderAnwarul Azim Majumder
Open Access
- 1 May 2024
- journal article
- research article
- Published by Taylor & Francis in Advances in Medical Education and Practice
- Vol. ume 15, 393-400
Abstract
Introduction: This research investigated the capabilities of ChatGPT-4 compared to medical students in answering MCQs using the revised Bloom’s Taxonomy as a benchmark. Methods: A cross-sectional study was conducted at The University of the West Indies, Barbados. ChatGPT-4 and medical students were assessed on MCQs from various medical courses using computer-based testing. Results: The study included 304 MCQs. Students demonstrated good knowledge, with 78% correctly answering at least 90% of the questions. However, ChatGPT-4 achieved a higher overall score (73.7%) compared to students (66.7%). Course type significantly affected ChatGPT-4’s performance, but revised Bloom’s Taxonomy levels did not. A detailed association check between program levels and Bloom’s taxonomy levels for correct answers by ChatGPT-4 showed a highly significant correlation (p< 0.001), reflecting a concentration of “remember-level” questions in preclinical and “evaluate-level” questions in clinical courses. Discussion: The study highlights ChatGPT-4’s proficiency in standardized tests but indicates limitations in clinical reasoning and practical skills. This performance discrepancy suggests that the effectiveness of artificial intelligence (AI) varies based on course content. Conclusion: While ChatGPT-4 shows promise as an educational tool, its role should be supplementary, with strategic integration into medical education to leverage its strengths and address limitations. Further research is needed to explore AI’s impact on medical education and student performance across educational levels and courses.Keywords
This publication has 33 references indexed in Scilit:
- ChatGPT performance on multiple choice question examinations in higher education. A pragmatic scoping reviewAssessment & Evaluation in Higher Education, 2024
- Assessment of the capacity of ChatGPT as a self-learning tool in medical pharmacology: a study using MCQsBMC Medical Education, 2023
- AI in medical education: medical student perception, curriculum recommendations and design suggestionsBMC Medical Education, 2023
- Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessmentsScientific Reports, 2023
- Perception, performance, and detectability of conversational artificial intelligence across 32 university coursesScientific Reports, 2023
- Application of ChatGPT in multilingual medical education: How does ChatGPT fare in 2023's Iranian residency entrance examinationInformatics in Medicine Unlocked, 2023
- ChatGPT in medical school: how successful is AI in progress testing?Medical Education Online, 2023
- Factors Influencing Quality of Team Discussion: Discourse Analysis in an Undergraduate Team-Based Learning Biology CourseCBE—Life Sciences Education, 2020
- Assessment in Medical EducationNew England Journal of Medicine, 2007
- Validity of final examinations in undergraduate medical trainingBMJ, 2000