Applications and Outcomes of Large‑Language‑Model‑Generated Feedback in Undergraduate Medical Education: A Scoping Review
Authors:
Top Cited Papers
- 6 January 2026
- journal article
- review article
- Published by Springer Nature in Medical Science Educator
- Vol. 36 (1) , 81-99
Abstract
Background Large language models (LLMs) are increasingly integrated into undergraduate medical education, particularly for generating learner feedback. While early LLM studies show promise, their educational impact and usage patterns remain unclear. The objective of this study is to systematically map how LLMs are being used to generate feedback for undergraduate medical students and to examine reported educational outcomes. Methods A scoping review was conducted following Arksey and O’Malley’s framework and reported using PRISMA-ScR. We searched PubMed and Web of Science and identified 4325 records. After screening/review, 42 studies were included. Data were charted using a structured form and outcomes were classified using Kirkpatrick Levels. Results The 42 included studies originated mostly from Global North countries, with nearly all using OpenAI’s GPT models. Feedback was delivered in two main contexts: simulated clinical encounters and text-based assessment tasks. Only 8 studies (19%) used randomized controlled trial designs. Educational outcomes were: 22 studies (52%) included no student data (Level 0); 10 reported student reaction (Level 1); 10 assessed learning gains (Level 2); none addressed behavior change or patient-level effects (Levels 3–4). LLM-generated feedback often matched expert feedback in short-term effectiveness but showed variable accuracy. Conclusions LLM-generated feedback is being explored across a range of educational settings, showing early signs of feasibility and perceived utility. However, the evidence base is limited in rigor and generalizability. Future research should assess behavioral and patient-level outcomes.Keywords
This publication has 62 references indexed in Scilit:
- Generative AI as the First Patient: Practice, Feedback, and ConfidenceMedical Science Educator, 2025
- Exploration of the Role of ChatGPT in Teaching Communication Skills for Medical Students: A Pilot StudyMedical Science Educator, 2025
- Development of a Universal Prompt as a Scalable Generative AI-Assisted Tool for USMLE Step 1 Style Multiple-Choice Question Refinement in Medical EducationMedical Science Educator, 2025
- Exploring Medical Student Trust in Generative Artificial Intelligence (ChatGPT) Versus Peers in Team-Based LearningMedical Science Educator, 2025
- AI Patient Actor: An Open-Access Generative-AI App for Communication Training in Health ProfessionsMedical Science Educator, 2024
- Comparing Scoring Consistency of Large Language Models with Faculty for Formative Assessments in Medical EducationJournal of General Internal Medicine, 2024
- Teaching humanism with humanoid: evaluating the potential of ChatGPT-4 as a pedagogical tool in bioethics education using validated clinical case vignettesInternational Journal of Ethics Education, 2024
- The leaky pipeline of publications and knowledge generation in medical educationPerspectives on Medical Education, 2022
- Scoping reviews in health professions education: challenges, considerations and lessons learned about epistemology and methodologyAdvances in Health Sciences Education, 2019
- Reflections on experimental research in medical educationAdvances in Health Sciences Education, 2008