What counts as a relevant gesture in the study of multimodal event expressions?
Open Access
- 31 December 2024
- journal article
- Published by John Benjamins Publishing Company in Gesture
- Vol. 23 (3) , 217-258
Abstract
This article explores how methodological decisions about which gestures to include impact the analysis of multimodal event representation. Gestures commonly temporally align with semantically co-expressive speech. However, when we consider not single concepts but whole events the picture gets more complicated, since events are not mapped onto single lexical items but across larger stretches of spoken language. To research gesture-speech integration in multimodal event expressions we must decide which gestures are considered ‘relevant’. Using caused motion events as a case study, we apply different definitions of gesture inclusion relating to temporal and semantic alignment. We investigate the impact of definition choice on the results by exploring corpus data from Australian English and two endangered Oceanic languages, Saliba-Logea and Sudest. The study shows comparable trends across some conditions but also language- and definition-specific differences. This has a bearing on how confidently we can compare studies based on different methodologies and/or languages.Keywords
This publication has 29 references indexed in Scilit:
- Cross-linguistic comparison of representations of motion in language and gestureGesture, 2012
- Language-specific encoding of placement events in gestures2010
- REPRESENTATION AND EMBODIMENT OF MEANING IN L2 COMMUNICATION: Motion Events in the Speech and Gesture of Advanced L2 Korean and L2 English SpeakersStudies in Second Language Acquisition, 2008
- How does linguistic framing of events influence co-speech gestures?: Insights from crosslinguistic variations and similaritiesGesture, 2005
- Gesture Visible Action as Utterance2004
- What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speakingJournal of Memory and Language, 2003
- Gesture, verb aspect, and the nature of iconic imagery in natural discourseGesture, 2002
- The production of gesture and speech2000
- Gesture, speech, and computational stages: A reply to McNeill.Psychological Review, 1989
- On some gestures' relation to talk1985