PRPatrick Louis Rohrer

Patrick Louis Rohrer

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Patrick Louis Rohrer has a total of 33 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • A temporal and pragmatic analysis of gesture-speech association : a corpus-based approach using the novel MultiModal MultiDimensional (M3D) labeling system

    by Patrick Rohrer

    Une analyse temporelle et pragmatique de l’association geste-parole : une approche basée sur un corpus utilisant le nouveau système d’annotation MultiModal MultiDimensionnel (M3D): Le langage est essentiellement multimodal. En effet, des études récentes ont montré à la fois la forte relation temporelle entre les gestes co-verbaux et la proéminence prosodique et leur pertinence pragmatique. Cependant, ces études ont eu tendance à se concentrer sur le rôle de la proéminence prosodique en tant qu'attracteur principal pour la production de gestes, et peu de recherches empiriques ont évalué d’une manière systématique le rôle de la structure phrastique prosodique, ou la contribution conjointe de la proéminence gestuelle et prosodique pour des effets pragmatiques, en particulier en tant comme marqueurs de la structure informationnelle. En outre, aucune étude n'a pris en compte la différence potentielle entre les gestes référentiels et les gestes non référentiels. Une analyse multidimensionnelle des différents traits du geste est cruciale pour permettre une évaluation systématique de leurs caractéristiques prosodiques et pragmatiques. Cette thèse poursuit un double objectif. Tout d'abord, elle propose une nouvelle approche de l'annotation des gestes co-verbaux qui épouse une vision dimensionnelle, selon laquelle les chercheurs devraient considérer les caractéristiques sémantiques, pragmatiques et prosodiques des gestes d'une manière non mutuellement exclusive. En second lieu, cette thèse vise à mieux comprendre les relations prosodiques et pragmatiques des gestes référentiels et non référentiels, en particulier la façon dont la structure phrastique prosodique influence les modèles de production gestuelle, et comment ces deux modes de communication interagissent pour des raisons pragmatiques.

  • Open Access

    Multidimensional Labeling of Gesture in Communication: the M3D Proposal

    by Patrick Louis Rohrer, Ulya Tütüncübasi, Júlia Florit-Pons, Ingrid Vilà-Giménez, Núria Esteve-Gibert, Ada Ren-Mitchell, Stefanie Shattuck-Hufnagel, Pilar Prieto

    Communication is multimodal in that speakers use not only their voices, but also co-speech gestures to communicate. Recent insights suggest that gestural behavior has a strong association with prosodic structure and that a single gesture can communicate various semantic and pragmatic meanings. This highlights the importance of developing a comprehensive, flexible, and transparent approach to gesture annotation that accounts for multiple dimensions of gesture, including a gesture’s form, prosodic properties, and semantic and pragmatic contributions. To address this need for an increasingly dimensionalized approach to multimodal data annotation, the main goal of this paper is to present and describe a novel labeling system for manual gestures. The MultiModal MultiDimensional (M3D) system consists of an open access package that has been developed in coordination with five different labs working on gesture and its interaction with speech. The package includes a set of reliable annotation guidelines, a validated training program, and two annotated audiovisual corpora that represent over 60 minutes of lecture-style speech.


  • Open Access

    From “I dance” to “she danced” with a flick of the hands: Audiovisual stress perception in Spanish

    by Patrick Louis Rohrer, Ronny Bujok, Lieke van Maastricht, Hans Rutger Bosker

    When talking, speakers naturally produce hand movements (co-speech gestures) that contribute to communication. Evidence in Dutch suggests that the timing of simple up-and-down, non-referential “beat” gestures influences spoken word recognition: the same auditory stimulus was perceived as CONtent (noun, capitalized letters indicate stressed syllables) when a beat gesture occurred on the first syllable, but as conTENT (adjective) when the gesture occurred on the second syllable. However, these findings were based on a small number of minimal pairs in Dutch, limiting the generalizability of the findings. We therefore tested this effect in Spanish, where lexical stress is highly relevant in the verb conjugation system, distinguishing bailo, “I dance” with word-initial stress from bailó, “she danced” with word-final stress. Testing a larger sample (N = 100), we also assessed whether individual differences in working memory capacity modulated how much individuals relied on the gestures in spoken word recognition. The results showed that, similar to Dutch, Spanish participants were biased to perceive lexical stress on the syllable that visually co-occurred with a beat gesture, with the effect being strongest when the acoustic stress cues were most ambiguous. No evidence was found for by-participant effect sizes to be influenced by individual differences in phonological or visuospatial working memory. These findings reveal gestural-speech coordination impacts lexical stress perception in a language where listeners are regularly confronted with such lexical stress contrasts, highlighting the impact of gestures’ timing on prominence perception and spoken word recognition.


  • A Roadmap for Technological Innovation in Multimodal Communication Research

    by Alina Gregori, Federica Amici, Ingmar Brilmayer, Aleksandra Ćwiek, Lennart Fritzsche, Susanne Fuchs, Alexander Henlein, Oliver Herbort, Frank Kügler, Jens Lemanski, et al.


  • Multimodal Development in Children's Narrative Speech: Evidence for Tight Gesture–Speech Temporal Alignment Patterns as Early as 5 Years Old

    by Júlia Florit-Pons, Ingrid Vilà-Giménez, Patrick Louis Rohrer, Pilar Prieto

    Purpose: This study aims to analyze the development of gesture–speech temporal alignment patterns in children's narrative speech from a longitudinal perspective and, specifically, the potential differences between different gesture types, namely, gestures that imagistically portray or refer to semantic content in speech (i.e., referential gestures) and those that lack semantic content (i.e., non-referential gestures). Method: This study uses an audiovisual corpus of narrative productions ( n = 332) from 83 children (43 girls, 40 boys) who participated in a narrative retelling task at two time points in development (at 5–6 and 7–9 years of age). The 332 narratives were coded for both manual co-speech gesture types and prosody. Gestural annotations included gesture phasing (i.e., preparation, stroke, hold, and recovery) and gesture types (in terms of referentiality, i.e., referential and non-referential), whereas prosodic annotations included pitch-accented syllables. Results: Results revealed that by ages 5–6 years, children already temporally aligned the stroke of both referential and non-referential gestures with pitch-accented syllables, showing no significant differences between these two gesture types. Conclusions: The results of the present study contribute to the view that both referential and non-referential gestures are aligned with pitch accentuation, and therefore, this is not only a characteristic of non-referential gestures. Our results also add support to McNeill's phonological synchronization rule from a developmental perspective and indirectly back up recent theories about the biomechanics of gesture–speech alignment, suggesting that this is an inherent ability of oral communication.