Multimodality matters in numerical communication
Authors:
Open Access
- 26 July 2023
- journal article
- review article
- Published by Frontiers Media SA in Frontiers in Psychology
- Vol. 14, 1130777
Abstract
Modern society depends on numerical information, which must be communicated accurately and effectively. Numerical communication is accomplished in different modalities—speech, writing, sign, gesture, graphs, and in naturally occurring settings it almost always involves more than one modality at once. Yet the modalities of numerical communication are often studied in isolation. Here we argue that, to understand and improve numerical communication, we must take seriously this multimodality. We first discuss each modality on its own terms, identifying their commonalities and differences. We then argue that numerical communication is shaped critically by interactions among modalities. We boil down these interactions to four types: one modality canamplifythe message of another; it candirectattention to content from another modality (e.g., using a gesture to guide attention to a relevant aspect of a graph); it canexplainanother modality (e.g., verbally explaining the meaning of an axis in a graph); and it canreinterpreta modality (e.g., framing an upwards-oriented trend as a bad outcome). We conclude by discussing how a focus on multimodality raises entirely new research questions about numerical communication.Keywords
This publication has 134 references indexed in Scilit:
- Truth Is at HandPsychological Science, 2010
- Gesturing Gives Children New Ideas About MathPsychological Science, 2009
- Beyond the number domainTrends in Cognitive Sciences, 2009
- Numerical thought with and without words: Evidence from indigenous Australian childrenProceedings of the National Academy of Sciences, 2008
- Gesturing makes learning lastCognition, 2008
- Sometimes area counts more than numberProceedings of the National Academy of Sciences, 2006
- Biological robustnessNature Reviews Genetics, 2004
- The mental representation of parity and number magnitude.Journal of Experimental Psychology: General, 1993
- An Information-Processing Analysis of Graph PerceptionJournal of the American Statistical Association, 1987
- Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical MethodsJournal of the American Statistical Association, 1984