Aslı Özyürek
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Aslı Özyürek has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Esam Ghaleb
Max Planck Institute for Psycholinguistics
2 shared publications - Judith Holler
Max Planck Institute for Psycholinguistics
2 shared publications - Marlou Rasenberg
Meertens Institute, Netherlands
2 shared publications - Wim Pouw
Department of Computational Cognitive Science, Research Center for Cognitive Science and Artificial Intelligence Tilburg University Tilburg the Netherlands
2 shared publications - Bulat Khaertdinov
Maastricht University Maastricht Netherlands
1 shared publication - Chinmaya Mishra
Max Planck Institute for Psycholinguistics
1 shared publication
Latest Publications
- Open Access
- 12 October 2026
- Published by Walter de Gruyter GmbH in Cognitive Linguistics
Investigating the role of pragmatic visual signals in language emergence using communication games and agent-based models
Studying the emergence of language has often relied on communication games and agent-based models, with a strong focus on referential linguistic signals, often overlooking pragmatic signals that are frequently expressed visually, such as facial signals. These signals are a crucial component of natural interaction, supporting repair, coordination, and negotiation of meaning. We outline existing language emergence experiments, highlighting findings from communication games and research on pragmatic facial signals in interaction, demonstrating their relevance to the emergence of communication systems. Furthermore, computational models of language emergence, such as agent-based models, largely overlook that communication is layered across multiple communication channels. We argue that incorporating pragmatic visual signals into these approaches can help to provide a more nuanced understanding of the processes underlying language emergence in natural, multimodal interaction. Following these insights, we propose two complementary directions for future research. First, communication game paradigms can be adapted to systematically analyze visual pragmatic signals, linking them to existing commonly used measures such as convergence, communicative success, and adaptation of linguistic forms. Second, agent-based models can be extended to multi-channel frameworks in which agents produce and interpret both referential and pragmatic signals, enabling investigation of how layered multimodal information shapes language emergence.
- 12 October 2026
- Published by Walter de Gruyter GmbH in Cognitive Linguistics
Open AccessInvestigating the role of pragmatic visual signals in language emergence using communication games and agent-based models
Studying the emergence of language has often relied on communication games and agent-based models, with a strong focus on referential linguistic signals, often overlooking pragmatic signals that are frequently expressed visually, such as facial signals. These signals are a crucial component of natural interaction, supporting repair, coordination, and negotiation of meaning. We outline existing language emergence experiments, highlighting findings from communication games and research on pragmatic facial signals in interaction, demonstrating their relevance to the emergence of communication systems. Furthermore, computational models of language emergence, such as agent-based models, largely overlook that communication is layered across multiple communication channels. We argue that incorporating pragmatic visual signals into these approaches can help to provide a more nuanced understanding of the processes underlying language emergence in natural, multimodal interaction. Following these insights, we propose two complementary directions for future research. First, communication game paradigms can be adapted to systematically analyze visual pragmatic signals, linking them to existing commonly used measures such as convergence, communicative success, and adaptation of linguistic forms. Second, agent-based models can be extended to multi-channel frameworks in which agents produce and interpret both referential and pragmatic signals, enabling investigation of how layered multimodal information shapes language emergence.
- Open Access
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 515-524
GroundingGaze: An Interactive Pipeline for Automatic Gaze Annotation
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 515-524
Open AccessGroundingGaze: An Interactive Pipeline for Automatic Gaze Annotation
- 10 September 2026
- Published by Springer Nature
- p. 528-545
SemConFlow: Semantic Grounding of Holistic Co-Speech Gesture Generation with Contrastive Flow-Matching
- 10 September 2026
- Published by Springer Nature
- p. 528-545
SemConFlow: Semantic Grounding of Holistic Co-Speech Gesture Generation with Contrastive Flow-Matching
- Open Access
- 4 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 274-283
Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluation
In face-to-face dialogues, the form-meaning relationship of co-speech gestures varies depending on contextual factors such as what the gestures refer to and the individual characteristics of speakers. These factors make co-speech gesture representation learning challenging. How can we learn meaningful gestures representations considering gestures’ variability and relationship with speech? This paper tackles this challenge by employing self-supervised contrastive learning techniques to learn gesture representations from skeletal and speech information. We propose an approach that includes both unimodal and multimodal pre-training to ground gesture representations in co-occurring speech. For training, we utilize a face-to-face dialogue dataset rich with representational iconic gestures. We conduct thorough intrinsic evaluations of the learned representations through comparison with human-annotated pairwise gesture similarity. Moreover, we perform a diagnostic probing analysis to assess the possibility of recovering interpretable gesture features from the learned representations. Our results show a significant positive correlation with human-annotated gesture similarity and reveal that the similarity between the learned representations is consistent with well-motivated patterns related to the dynamics of dialogue interaction. Moreover, our findings demonstrate that several features concerning the form of gestures can be recovered from the latent representations. Overall, this study shows that multimodal contrastive learning is a promising approach for learning gesture representations, which opens the door to using such representations in larger-scale gesture analysis studies.
- 4 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 274-283
Open AccessLearning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluation
In face-to-face dialogues, the form-meaning relationship of co-speech gestures varies depending on contextual factors such as what the gestures refer to and the individual characteristics of speakers. These factors make co-speech gesture representation learning challenging. How can we learn meaningful gestures representations considering gestures’ variability and relationship with speech? This paper tackles this challenge by employing self-supervised contrastive learning techniques to learn gesture representations from skeletal and speech information. We propose an approach that includes both unimodal and multimodal pre-training to ground gesture representations in co-occurring speech. For training, we utilize a face-to-face dialogue dataset rich with representational iconic gestures. We conduct thorough intrinsic evaluations of the learned representations through comparison with human-annotated pairwise gesture similarity. Moreover, we perform a diagnostic probing analysis to assess the possibility of recovering interpretable gesture features from the learned representations. Our results show a significant positive correlation with human-annotated gesture similarity and reveal that the similarity between the learned representations is consistent with well-motivated patterns related to the dynamics of dialogue interaction. Moreover, our findings demonstrate that several features concerning the form of gestures can be recovered from the latent representations. Overall, this study shows that multimodal contrastive learning is a promising approach for learning gesture representations, which opens the door to using such representations in larger-scale gesture analysis studies.
- Top Cited PapersOpen Access
- 9 November 2022
- Published by Springer Nature in Scientific Reports
- Vol. 12 (1) , 1-11
The multimodal nature of communicative efficiency in social interaction
How does communicative efficiency shape language use? We approach this question by studying it at the level of the dyad, and in terms of multimodal utterances. We investigate whether and how people minimize their joint speech and gesture efforts in face-to-face interactions, using linguistic and kinematic analyses. We zoom in on other-initiated repair—a conversational microcosm where people coordinate their utterances to solve problems with perceiving or understanding. We find that efforts in the spoken and gestural modalities are wielded in parallel across repair turns of different types, and that people repair conversational problems in the most cost-efficient way possible, minimizing the joint multimodal effort for the dyad as a whole. These results are in line with the principle of least collaborative effort in speech and with the reduction of joint costs in non-linguistic joint actions. The results extend our understanding of those coefficiency principles by revealing that they pertain to multimodal utterance design.
- 9 November 2022
- Published by Springer Nature in Scientific Reports
- Vol. 12 (1) , 1-11
Top Cited PapersOpen AccessThe multimodal nature of communicative efficiency in social interaction
How does communicative efficiency shape language use? We approach this question by studying it at the level of the dyad, and in terms of multimodal utterances. We investigate whether and how people minimize their joint speech and gesture efforts in face-to-face interactions, using linguistic and kinematic analyses. We zoom in on other-initiated repair—a conversational microcosm where people coordinate their utterances to solve problems with perceiving or understanding. We find that efforts in the spoken and gestural modalities are wielded in parallel across repair turns of different types, and that people repair conversational problems in the most cost-efficient way possible, minimizing the joint multimodal effort for the dyad as a whole. These results are in line with the principle of least collaborative effort in speech and with the reduction of joint costs in non-linguistic joint actions. The results extend our understanding of those coefficiency principles by revealing that they pertain to multimodal utterance design.
- Open Access
- 23 August 2021
- Published by Ubiquity Press, Ltd. in Journal of Cognition
- Vol. 4 (1)
Considering the Nature of Multimodal Language from a Crosslinguistic Perspective
Language in its primary face-to-face context is multimodal (e.g., Holler and Levinson, 2019; Perniss, 2018). Thus, understanding how expressions in the vocal and visual modalities together contribute to our notions of language structure, use, processing, and transmission (i.e., acquisition, evolution, emergence) in different languages and cultures should be a fundamental goal of language sciences. This requires a new framework of language that brings together how arbitrary and non-arbitrary and motivated semiotic resources of language relate to each other. Current commentary evaluates such a proposal by Murgiano et al (2021) from a crosslinguistic perspective taking variation as well as systematicity in multimodal utterances into account.
- 23 August 2021
- Published by Ubiquity Press, Ltd. in Journal of Cognition
- Vol. 4 (1)
Open AccessConsidering the Nature of Multimodal Language from a Crosslinguistic Perspective
Language in its primary face-to-face context is multimodal (e.g., Holler and Levinson, 2019; Perniss, 2018). Thus, understanding how expressions in the vocal and visual modalities together contribute to our notions of language structure, use, processing, and transmission (i.e., acquisition, evolution, emergence) in different languages and cultures should be a fundamental goal of language sciences. This requires a new framework of language that brings together how arbitrary and non-arbitrary and motivated semiotic resources of language relate to each other. Current commentary evaluates such a proposal by Murgiano et al (2021) from a crosslinguistic perspective taking variation as well as systematicity in multimodal utterances into account.
- Open Access
- 31 December 2015
- Published by John Benjamins Publishing Company in Dutch Journal of Applied Linguistics
- Vol. 4 (2) , 222-240
Discourse management
Speakers achieve coherence in discourse by alternating between differential lexical forms e.g. noun phrase, pronoun, and null form in accordance with the accessibility of the entities they refer to, i.e. whether they introduce an entity into discourse for the first time or continue referring to an entity they already mentioned before. Moreover, tracking of entities in discourse is a multimodal phenomenon. Studies show that speakers are sensitive to the informational structure of discourse and use fuller forms (e.g. full noun phrases) in speech and gesture more when re-introducing an entity while they use attenuated forms (e.g. pronouns) in speech and gesture less when maintaining a referent. However, those studies focus mainly on non-pro-drop languages (e.g. English, German and French). The present study investigates whether the same pattern holds for pro-drop languages. It draws data from adult native speakers of Turkish using elicited narratives. We find that Turkish speakers mostly use fuller forms to code subject referents in re-introduction context and the null form in maintenance context and they point to gesture space for referents more in re-introduction context compared maintained context. Hence we provide supportive evidence for the reverse correlation between the accessibility of a discourse referent and its coding in speech and gesture. We also find that, as a novel contribution, third person pronoun is used in re-introduction context only when the referent was previously mentioned as the object argument of the immediately preceding clause.
- 31 December 2015
- Published by John Benjamins Publishing Company in Dutch Journal of Applied Linguistics
- Vol. 4 (2) , 222-240
Open AccessDiscourse management
Speakers achieve coherence in discourse by alternating between differential lexical forms e.g. noun phrase, pronoun, and null form in accordance with the accessibility of the entities they refer to, i.e. whether they introduce an entity into discourse for the first time or continue referring to an entity they already mentioned before. Moreover, tracking of entities in discourse is a multimodal phenomenon. Studies show that speakers are sensitive to the informational structure of discourse and use fuller forms (e.g. full noun phrases) in speech and gesture more when re-introducing an entity while they use attenuated forms (e.g. pronouns) in speech and gesture less when maintaining a referent. However, those studies focus mainly on non-pro-drop languages (e.g. English, German and French). The present study investigates whether the same pattern holds for pro-drop languages. It draws data from adult native speakers of Turkish using elicited narratives. We find that Turkish speakers mostly use fuller forms to code subject referents in re-introduction context and the null form in maintenance context and they point to gesture space for referents more in re-introduction context compared maintained context. Hence we provide supportive evidence for the reverse correlation between the accessibility of a discourse referent and its coding in speech and gesture. We also find that, as a novel contribution, third person pronoun is used in re-introduction context only when the referent was previously mentioned as the object argument of the immediately preceding clause.
- Open Access
- 6 June 2007
- Published by John Benjamins Publishing Company
- p. 67-74
5 How does Spoken Language Shape Iconic Gestures?
- 6 June 2007
- Published by John Benjamins Publishing Company
- p. 67-74
Open Access5 How does Spoken Language Shape Iconic Gestures?