Multimedia learning with multiple modalities beyond hearing and sight
Authors:
Open Access
- 18 September 2026
- journal article
- review article
- Published by Frontiers Media SA in Frontiers in Psychology
- Vol. 17, 1433994
Abstract
Education and educational research fundamentally rely on advancing learning via multiple representations and comprehensive technological developments that address various sensory modalities. For a more profound understanding of learning processes associated with multimodal learning, psychological theories and models are essential. For the last two decades, most of these processes have been traced to the cognitive theory of multimedia learning and the integrated model of text and picture comprehension, both of which focus on bimodal learning with words and static or animated pictures. However, the field lacks a more comprehensive theoretical approach and more empirical evidence for principles of multimedia learning that address multimedia learning beyond common representations and modalities. Therefore, we propose implications for expanding existing theories by describing learning that involves multiple external representations and addresses various sensory modalities beyond just hearing and sight. In addition, for this theoretical extension, we derive implications for expanded and adapted principles of multimedia learning that include further sensory modalities. Overall, the extension and implications presented here represent drivers for future research on multimodal learning with multiple representations.Keywords
This publication has 110 references indexed in Scilit:
- The Somatosensory System2020
- Multimediales Lernen: Lehren und Lernen mit Texten und Bildern2018
- Representational Competence, Understanding of Experiments, Phenomena and Basic Concepts in Geometrical Optics: A Representational Approach2017
- Learning By Tracing Worked ExamplesApplied Cognitive Psychology, 2015
- Students’ Learning Strategies With Multiple Representations: Explanations of the Human Breathing MechanismScience Education, 2014
- Gestures in Instructional Animations: A Helping Hand to Understanding Non‐human Movements?Applied Cognitive Psychology, 2013
- Making Better Use of Multiple Representations: How Fostering Metacognition Can Help2013
- Reading information graphics: The role of spatial contiguity and dual attentional guidanceApplied Cognitive Psychology, 2008
- Dual coding theory and educationEducational Psychology Review, 1991
- Encoding and retaining information in the visuals and verbals of an educational movieECTJ, 1983