AI-Powered Simulated Patients with Automated Feedback for Enhancing Headache History-Taking Skills: A Convergent Mixed-Methods Study
Authors:
Preprint
- 9 February 2026
- preprint
- Published by Springer Nature in Research Square
Abstract
Background Medical students often struggle with headache disorders, which are common but diagnostically challenging due to limited clinical exposure and feedback during early clinical training. This study evaluated the feasibility and effectiveness of an AI-powered simulated patient (AISP) chatbot in enhancing history-taking and clinical reasoning for three primary headache disorders—migraine, tension-type, and cluster headache—among medical students at a tertiary university hospital in Thailand. Methods We developed a simulated patient platform that provided automated, rubric-based feedback across three headache cases. Third-year medical students who had completed pre-clerkship requirements participated in a mixed-methods experimental study with a pre-post design. The study consisted of a 1-week self-study phase followed by a pre-test Objective Structured Clinical Examination (OSCE), a 1-week washout period, and a 1-week AISP practice phase followed by a post-test OSCE. Quantitative outcomes included changes in OSCE scores, Clinical Reasoning Indicator-History Taking (CRI-HT) scores, usability/satisfaction measured using the Chatbot Usability Questionnaire (CUQ). Focus group discussions (FGDs) explored participants’ learning experiences, perceived system limitations, and recommendations for future implementation. Qualitative data were analyzed thematically. Results Twenty-five medical students participated. OSCE scores increased by 22.5 points ( p < 0.001, Cohen’s d = 1.93), and CRI-HT scores increased by 8.8 points ( p < 0.001, Cohen’s d = 1.81). CUQ findings indicated high usability. Students commonly cited the platform’s intuitive interface and informative feedback, although some noted the chatbot’s responses were overly robotic. FGDs further highlighted three themes: (1) convenient, structured practice support learning; (2) dialogue felt realistic but was limited for practicing communication skills; and (3) students recommended greater case variety and integration into the curriculum. Conclusions An AISP platform with automated, rubric-based feedback was feasible and associated with improvements in headache history-taking and clinical reasoning. It serves as a valuable complement to traditional teaching by enabling structured, repeated practice with timely feedback.Keywords
All Related Versions
- Version 1, 2026-02-09, Research Square
- Published version: BMC Medical Education, 26 (1).
This publication has 20 references indexed in Scilit:
- Qualitative exploration of digital chatbot use in medical education: A pilot studyDigital Health, 2021
- Usability testing of a healthcare chatbot: Can we use conventional methods to assess conversational user interfaces?2019
- Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017The Lancet, 2018
- Headache Classification Committee of the International Headache Society (IHS) The International Classification of Headache Disorders, 3rd editionCephalalgia, 2018
- Medical Undergraduate Survey on Headache Education in Singapore: Knowledge, Perceptions, and Assessment of Unmet NeedsHeadache: The Journal of Head and Face Pain, 2017
- Headache Education in Adult Neurology Residency: A Survey of Program Directors and Chief ResidentsHeadache: The Journal of Head and Face Pain, 2016
- Simulation and its role in medical educationMedical Journal Armed Forces India, 2012
- The use of simulated patients in medical education: AMEE Guide No 42Medical Teacher, 2009
- Improving Medical Students’ Success in Promoting Health Behavior Change: A Curriculum EvaluationJournal of General Internal Medicine, 2008
- American Academic Headache Specialists in Neurology: Practice Characteristics and CultureCephalalgia, 2004