Investigating the role of pragmatic visual signals in language emergence using communication games and agent-based models
Open Access
- 12 October 2026
- journal article
- Published by Walter de Gruyter GmbH in Cognitive Linguistics
Abstract
Studying the emergence of language has often relied on communication games and agent-based models, with a strong focus on referential linguistic signals, often overlooking pragmatic signals that are frequently expressed visually, such as facial signals. These signals are a crucial component of natural interaction, supporting repair, coordination, and negotiation of meaning. We outline existing language emergence experiments, highlighting findings from communication games and research on pragmatic facial signals in interaction, demonstrating their relevance to the emergence of communication systems. Furthermore, computational models of language emergence, such as agent-based models, largely overlook that communication is layered across multiple communication channels. We argue that incorporating pragmatic visual signals into these approaches can help to provide a more nuanced understanding of the processes underlying language emergence in natural, multimodal interaction. Following these insights, we propose two complementary directions for future research. First, communication game paradigms can be adapted to systematically analyze visual pragmatic signals, linking them to existing commonly used measures such as convergence, communicative success, and adaptation of linguistic forms. Second, agent-based models can be extended to multi-channel frameworks in which agents produce and interpret both referential and pragmatic signals, enabling investigation of how layered multimodal information shapes language emergence.Keywords
This publication has 65 references indexed in Scilit:
- Stream noise induces song plasticity and a shift to visual signals in a riverine songbirdCurrent Biology, 2025
- Facial clues to conversational intentionsTrends in Cognitive Sciences, 2025
- Emergent naming conventions in a foraging robot swarmSwarm Intelligence, 2022
- Evolving artificial sign languages in the lab: From improvised gesture to systematic signCognition, 2019
- Compositional structure can emerge without generational transmissionCognition, 2019
- Facial Displays Are Tools for Social InfluenceTrends in Cognitive Sciences, 2018
- Compression and communication in the cultural evolution of linguistic structureCognition, 2015
- Modeling the cultural evolution of languagePhysics of Life Reviews, 2011
- Gesturing on the telephone: Independent effects of dialogue and visibilityJournal of Memory and Language, 2008
- Audiovisual prosody and feeling of knowingJournal of Memory and Language, 2005