"A Safe First Step": Design and Evaluation of an Emotionally Expressive AI Virtual Patient for Clinical Simulation in Speech-Language Pathology Training
Authors:
- Yuqi HuYHYuqi Hu
- Yujing WangYWYujing Wang
- Brandon WatanabeBWBrandon Watanabe
- Mingkai GaoMGMingkai Gao
- Zhenzhen QinZQZhenzhen Qin
- Kunyi ShiKSKunyi Shi
- Zinan ZhangZZZinan Zhang
- Shreevidhya ShambannaSSShreevidhya Shambanna
- Zhuoying XueZXZhuoying Xue
- Tao ZouTZTao Zou
- Qiwen XiongQXQiwen Xiong
- Nia JohnsonNJNia Johnson
Preprint
- 4 February 2026
- preprint
- Published by Springer Nature in Research Square
Abstract
Speech-language pathology (SLP) training programs face persistent challenges in providing sufficient opportunities for clinical placements or simulations that allow students to practice clinical communication with patients with communication disorders. We designed and implemented V.O.I.C.E., an AI-powered virtual patient system that enables realistic, open-ended clinical simulation for SLP students. V.O.I.C.E. integrates (1) a large language model (LLM) role-play agent designed to produce responses consistent with symptoms of post-stroke expressive aphasia (Broca’s aphasia), (2) a multimodal emotional expression pipeline that renders context-appropriate emotions of the virtual patient through coordinated speech prosody, facial expressions, and gestures, and (3) an LLM-based, rubric-guided debriefing module that generates structured formative feedback across core clinical communication competencies. We conducted a pilot mixed-methods, quasi-experimental evaluation with 11 graduate-level SLP students, where they completed three simulation sessions using the V.O.I.C.E system with progressive challenges. Results show significant gains in students' overall self-efficacy for interacting with Broca's aphasia patients in clinical settings. The self-efficacy gains were moderated by students’ perceptions of multimodal realism of the simulations. Qualitative findings highlight the system’s value as a low-stakes “bridge” for transition to clinical practice, which supports learning through iterative practice, reflection, and strategy refinement. This work provides a framework for AI-based simulation training applications that emphasizes interpersonal communication.Keywords
All Related Versions
This publication has 87 references indexed in Scilit:
- Training LLM-Based Tutors to Improve Student Learning Outcomes in Dialogues2025
- Enhancing clinical reasoning skills for medical students: a qualitative comparison of LLM-powered social robotic versus computer-based virtual patients within rheumatologyRheumatology International, 2024
- Multimodal Emotion Recognition in Learning Environments2023
- Association Between Emotional Disorders and Speech and Language Impairments: A National Population-Based StudyChild Psychiatry & Human Development, 2019
- Academic self-efficacy: from educational theory to instructional practicePerspectives on Medical Education, 2012
- Realism, authenticity, and learning in healthcare simulations: rules of relevance and irrelevance as interactive achievementsInstructional Science, 2012
- Sentic Avatar: Multimodal Affective Conversational Agent with Common Sense2011
- The Control-Value Theory of Achievement Emotions: Assumptions, Corollaries, and Implications for Educational Research and PracticeEducational Psychology Review, 2006
- Self-Efficacy: An Essential Motive to LearnContemporary Educational Psychology, 2000
- Formative assessment and the design of instructional systemsInstructional Science, 1989