Answering correctly or incorrectly: the impact of multimodal feedback from education robot voice gender and body movements on users’ learning outcomes
- 27 October 2025
- journal article
- Published by Taylor & Francis in Ergonomics
- Vol. 69 (11) , 2281-2294
Abstract
Despite the growing interest in multimodal communication in human-robot interaction (HRI), research about how to design robots' multimodal feedback in educational contexts remains limited. This study employed a 2 × 3 (voice gender: male vs female × body movements: head vs hand vs combined movements) within-subject experimental design to investigate the effects of multimodal feedback (voice and body movements) on learning outcomes from three perspectives: affective outcomes, visual attention, and cerebral activity. The results revealed that voice gender and body movements significantly influenced users' affective outcomes and cerebral activity. Voice gender impacted visual attention, and an interaction effect between voice gender and body movements was observed in cerebral activity. Moreover, this research suggested that robots with female voices and combined movements promoted learning outcomes and revealed that robot voice gender and body movements may independently shape users' learning outcomes in low-demand cognitive feedback.Keywords
Funding Information
- National Natural Science Foundation of China (72071035)
This publication has 47 references indexed in Scilit:
- The Past, Present, and Future of the Cognitive Theory of Multimedia LearningEducational Psychology Review, 2024
- Artificial Intelligence in Science Education (2013–2023): Research Trends in Ten YearsJournal of Science Education and Technology, 2023
- Effects of Robotics Education on Young Children’s Cognitive Development: a Pilot Study with Eye-TrackingJournal of Science Education and Technology, 2023
- Higher Nonverbal Immediacy Leads to Greater Learning Gains in Child-Robot Tutoring Interactions2015
- An introduction to multimodal communicationBehavioral Ecology and Sociobiology, 2013
- To Err is Human(-like): Effects of Robot Gesture on Perceived Anthropomorphism and LikabilityInternational Journal of Social Robotics, 2013
- Making Social Robots More Attractive: The Effects of Voice Pitch, Humor and EmpathyInternational Journal of Social Robotics, 2013
- Generation and Evaluation of Communicative Robot GestureInternational Journal of Social Robotics, 2012
- A Quantitative Comparison of Simultaneous BOLD fMRI and NIRS Recordings during Functional Brain ActivationNeuroImage, 2002
- System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transilluminationMedical & Biological Engineering & Computing, 1988