Patrick Louis Rohrer
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Patrick Louis Rohrer has a total of 33 co-authors. Here are the top co-authors by number of shared publications:
- Ingrid Vilà-Giménez
Department of Specific Didactics, University of Girona, Girona, Spain
2 shared publications - Júlia Florit-Pons
Department of Translation and Language Sciences, Universitat Pompeu Fabra, Barcelona, Spain
2 shared publications - Pilar Prieto
Institució Catalana de Recerca i Estudis Avançats
2 shared publications Ada Ren-Mitchell
Speech Communication Group, Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, USA
1 shared publication- Aleksandra Ćwiek
Center for Language Evolution Studies, Nicolaus Copernicus University in Toruń , Toruń , Poland
1 shared publication - Alexander Henlein
Text Technology Lab Goethe University Frankfurt Germany
1 shared publication
Latest Publications
- 7 April 2026
- Published by Agence Bibliographique de l'Enseignement Supérieur
A temporal and pragmatic analysis of gesture-speech association : a corpus-based approach using the novel MultiModal MultiDimensional (M3D) labeling system
Une analyse temporelle et pragmatique de l’association geste-parole : une approche basée sur un corpus utilisant le nouveau système d’annotation MultiModal MultiDimensionnel (M3D): Le langage est essentiellement multimodal. En effet, des études récentes ont montré à la fois la forte relation temporelle entre les gestes co-verbaux et la proéminence prosodique et leur pertinence pragmatique. Cependant, ces études ont eu tendance à se concentrer sur le rôle de la proéminence prosodique en tant qu'attracteur principal pour la production de gestes, et peu de recherches empiriques ont évalué d’une manière systématique le rôle de la structure phrastique prosodique, ou la contribution conjointe de la proéminence gestuelle et prosodique pour des effets pragmatiques, en particulier en tant comme marqueurs de la structure informationnelle. En outre, aucune étude n'a pris en compte la différence potentielle entre les gestes référentiels et les gestes non référentiels. Une analyse multidimensionnelle des différents traits du geste est cruciale pour permettre une évaluation systématique de leurs caractéristiques prosodiques et pragmatiques. Cette thèse poursuit un double objectif. Tout d'abord, elle propose une nouvelle approche de l'annotation des gestes co-verbaux qui épouse une vision dimensionnelle, selon laquelle les chercheurs devraient considérer les caractéristiques sémantiques, pragmatiques et prosodiques des gestes d'une manière non mutuellement exclusive. En second lieu, cette thèse vise à mieux comprendre les relations prosodiques et pragmatiques des gestes référentiels et non référentiels, en particulier la façon dont la structure phrastique prosodique influence les modèles de production gestuelle, et comment ces deux modes de communication interagissent pour des raisons pragmatiques.
- 7 April 2026
- Published by Agence Bibliographique de l'Enseignement Supérieur
A temporal and pragmatic analysis of gesture-speech association : a corpus-based approach using the novel MultiModal MultiDimensional (M3D) labeling system
Une analyse temporelle et pragmatique de l’association geste-parole : une approche basée sur un corpus utilisant le nouveau système d’annotation MultiModal MultiDimensionnel (M3D): Le langage est essentiellement multimodal. En effet, des études récentes ont montré à la fois la forte relation temporelle entre les gestes co-verbaux et la proéminence prosodique et leur pertinence pragmatique. Cependant, ces études ont eu tendance à se concentrer sur le rôle de la proéminence prosodique en tant qu'attracteur principal pour la production de gestes, et peu de recherches empiriques ont évalué d’une manière systématique le rôle de la structure phrastique prosodique, ou la contribution conjointe de la proéminence gestuelle et prosodique pour des effets pragmatiques, en particulier en tant comme marqueurs de la structure informationnelle. En outre, aucune étude n'a pris en compte la différence potentielle entre les gestes référentiels et les gestes non référentiels. Une analyse multidimensionnelle des différents traits du geste est cruciale pour permettre une évaluation systématique de leurs caractéristiques prosodiques et pragmatiques. Cette thèse poursuit un double objectif. Tout d'abord, elle propose une nouvelle approche de l'annotation des gestes co-verbaux qui épouse une vision dimensionnelle, selon laquelle les chercheurs devraient considérer les caractéristiques sémantiques, pragmatiques et prosodiques des gestes d'une manière non mutuellement exclusive. En second lieu, cette thèse vise à mieux comprendre les relations prosodiques et pragmatiques des gestes référentiels et non référentiels, en particulier la façon dont la structure phrastique prosodique influence les modèles de production gestuelle, et comment ces deux modes de communication interagissent pour des raisons pragmatiques.
- Open Access
- 26 June 2025
- Published by Springer Nature in Corpus Pragmatics
- Vol. 9 (4) , 411-433
Multidimensional Labeling of Gesture in Communication: the M3D Proposal
Communication is multimodal in that speakers use not only their voices, but also co-speech gestures to communicate. Recent insights suggest that gestural behavior has a strong association with prosodic structure and that a single gesture can communicate various semantic and pragmatic meanings. This highlights the importance of developing a comprehensive, flexible, and transparent approach to gesture annotation that accounts for multiple dimensions of gesture, including a gesture’s form, prosodic properties, and semantic and pragmatic contributions. To address this need for an increasingly dimensionalized approach to multimodal data annotation, the main goal of this paper is to present and describe a novel labeling system for manual gestures. The MultiModal MultiDimensional (M3D) system consists of an open access package that has been developed in coordination with five different labs working on gesture and its interaction with speech. The package includes a set of reliable annotation guidelines, a validated training program, and two annotated audiovisual corpora that represent over 60 minutes of lecture-style speech.
- 26 June 2025
- Published by Springer Nature in Corpus Pragmatics
- Vol. 9 (4) , 411-433
Open AccessMultidimensional Labeling of Gesture in Communication: the M3D Proposal
Communication is multimodal in that speakers use not only their voices, but also co-speech gestures to communicate. Recent insights suggest that gestural behavior has a strong association with prosodic structure and that a single gesture can communicate various semantic and pragmatic meanings. This highlights the importance of developing a comprehensive, flexible, and transparent approach to gesture annotation that accounts for multiple dimensions of gesture, including a gesture’s form, prosodic properties, and semantic and pragmatic contributions. To address this need for an increasingly dimensionalized approach to multimodal data annotation, the main goal of this paper is to present and describe a novel labeling system for manual gestures. The MultiModal MultiDimensional (M3D) system consists of an open access package that has been developed in coordination with five different labs working on gesture and its interaction with speech. The package includes a set of reliable annotation guidelines, a validated training program, and two annotated audiovisual corpora that represent over 60 minutes of lecture-style speech.
- Open Access
- 7 April 2025
- Published by Springer Nature in Psychonomic Bulletin & Review
- Vol. 32 (5) , 2136-2145
From “I dance” to “she danced” with a flick of the hands: Audiovisual stress perception in Spanish
When talking, speakers naturally produce hand movements (co-speech gestures) that contribute to communication. Evidence in Dutch suggests that the timing of simple up-and-down, non-referential “beat” gestures influences spoken word recognition: the same auditory stimulus was perceived as CONtent (noun, capitalized letters indicate stressed syllables) when a beat gesture occurred on the first syllable, but as conTENT (adjective) when the gesture occurred on the second syllable. However, these findings were based on a small number of minimal pairs in Dutch, limiting the generalizability of the findings. We therefore tested this effect in Spanish, where lexical stress is highly relevant in the verb conjugation system, distinguishing bailo, “I dance” with word-initial stress from bailó, “she danced” with word-final stress. Testing a larger sample (N = 100), we also assessed whether individual differences in working memory capacity modulated how much individuals relied on the gestures in spoken word recognition. The results showed that, similar to Dutch, Spanish participants were biased to perceive lexical stress on the syllable that visually co-occurred with a beat gesture, with the effect being strongest when the acoustic stress cues were most ambiguous. No evidence was found for by-participant effect sizes to be influenced by individual differences in phonological or visuospatial working memory. These findings reveal gestural-speech coordination impacts lexical stress perception in a language where listeners are regularly confronted with such lexical stress contrasts, highlighting the impact of gestures’ timing on prominence perception and spoken word recognition.
- 7 April 2025
- Published by Springer Nature in Psychonomic Bulletin & Review
- Vol. 32 (5) , 2136-2145
Open AccessFrom “I dance” to “she danced” with a flick of the hands: Audiovisual stress perception in Spanish
When talking, speakers naturally produce hand movements (co-speech gestures) that contribute to communication. Evidence in Dutch suggests that the timing of simple up-and-down, non-referential “beat” gestures influences spoken word recognition: the same auditory stimulus was perceived as CONtent (noun, capitalized letters indicate stressed syllables) when a beat gesture occurred on the first syllable, but as conTENT (adjective) when the gesture occurred on the second syllable. However, these findings were based on a small number of minimal pairs in Dutch, limiting the generalizability of the findings. We therefore tested this effect in Spanish, where lexical stress is highly relevant in the verb conjugation system, distinguishing bailo, “I dance” with word-initial stress from bailó, “she danced” with word-final stress. Testing a larger sample (N = 100), we also assessed whether individual differences in working memory capacity modulated how much individuals relied on the gestures in spoken word recognition. The results showed that, similar to Dutch, Spanish participants were biased to perceive lexical stress on the syllable that visually co-occurred with a beat gesture, with the effect being strongest when the acoustic stress cues were most ambiguous. No evidence was found for by-participant effect sizes to be influenced by individual differences in phonological or visuospatial working memory. These findings reveal gestural-speech coordination impacts lexical stress perception in a language where listeners are regularly confronted with such lexical stress contrasts, highlighting the impact of gestures’ timing on prominence perception and spoken word recognition.
- 9 July 2023
- Published by Springer Nature
- p. 402-438
A Roadmap for Technological Innovation in Multimodal Communication Research
- 9 July 2023
- Published by Springer Nature
- p. 402-438
A Roadmap for Technological Innovation in Multimodal Communication Research
- 7 March 2023
- Published by American Speech Language Hearing Association in Journal of Speech, Language, and Hearing Research
- Vol. 66 (3) , 888-900
Multimodal Development in Children's Narrative Speech: Evidence for Tight Gesture–Speech Temporal Alignment Patterns as Early as 5 Years Old
Purpose: This study aims to analyze the development of gesture–speech temporal alignment patterns in children's narrative speech from a longitudinal perspective and, specifically, the potential differences between different gesture types, namely, gestures that imagistically portray or refer to semantic content in speech (i.e., referential gestures) and those that lack semantic content (i.e., non-referential gestures). Method: This study uses an audiovisual corpus of narrative productions ( n = 332) from 83 children (43 girls, 40 boys) who participated in a narrative retelling task at two time points in development (at 5–6 and 7–9 years of age). The 332 narratives were coded for both manual co-speech gesture types and prosody. Gestural annotations included gesture phasing (i.e., preparation, stroke, hold, and recovery) and gesture types (in terms of referentiality, i.e., referential and non-referential), whereas prosodic annotations included pitch-accented syllables. Results: Results revealed that by ages 5–6 years, children already temporally aligned the stroke of both referential and non-referential gestures with pitch-accented syllables, showing no significant differences between these two gesture types. Conclusions: The results of the present study contribute to the view that both referential and non-referential gestures are aligned with pitch accentuation, and therefore, this is not only a characteristic of non-referential gestures. Our results also add support to McNeill's phonological synchronization rule from a developmental perspective and indirectly back up recent theories about the biomechanics of gesture–speech alignment, suggesting that this is an inherent ability of oral communication.
- 7 March 2023
- Published by American Speech Language Hearing Association in Journal of Speech, Language, and Hearing Research
- Vol. 66 (3) , 888-900
Multimodal Development in Children's Narrative Speech: Evidence for Tight Gesture–Speech Temporal Alignment Patterns as Early as 5 Years Old
Purpose: This study aims to analyze the development of gesture–speech temporal alignment patterns in children's narrative speech from a longitudinal perspective and, specifically, the potential differences between different gesture types, namely, gestures that imagistically portray or refer to semantic content in speech (i.e., referential gestures) and those that lack semantic content (i.e., non-referential gestures). Method: This study uses an audiovisual corpus of narrative productions ( n = 332) from 83 children (43 girls, 40 boys) who participated in a narrative retelling task at two time points in development (at 5–6 and 7–9 years of age). The 332 narratives were coded for both manual co-speech gesture types and prosody. Gestural annotations included gesture phasing (i.e., preparation, stroke, hold, and recovery) and gesture types (in terms of referentiality, i.e., referential and non-referential), whereas prosodic annotations included pitch-accented syllables. Results: Results revealed that by ages 5–6 years, children already temporally aligned the stroke of both referential and non-referential gestures with pitch-accented syllables, showing no significant differences between these two gesture types. Conclusions: The results of the present study contribute to the view that both referential and non-referential gestures are aligned with pitch accentuation, and therefore, this is not only a characteristic of non-referential gestures. Our results also add support to McNeill's phonological synchronization rule from a developmental perspective and indirectly back up recent theories about the biomechanics of gesture–speech alignment, suggesting that this is an inherent ability of oral communication.