Cans and cants: Computational potentials for multimodality with a case study in head position
Authors:
- 25 November 2016
- journal article
- research article
- Published by Wiley in Journal of Sociolinguistics
- Vol. 20 (5) , 677-711
Abstract
As the study of embodiment and multimodality in interaction grows in importance, there is a need for novel methodological approaches to understand how multimodal variables pattern together along social and contextual lines, and how they systematically coalesce in communicative meanings. In this work, we propose to adopt computational tools to generate replicable annotations of bodily variables: these can be examined statistically to understand their patterning with other variables across diverse speakers and interactional contexts, and can help organize qualitative analyses of large datasets. We demonstrate the possibilities thereby with a case study in head cant (side‐to‐side tilt of the head) in a dataset of video blogs and laboratory‐collected interactions, computationally extracting cant and prosody from video and audio and analyzing their interactions, looking at gender in particular. We find that head cant indexes an orientation towards the interlocutor and a sense of shared understanding, can serve a ‘bracketing’ function in interaction (for speakers to create parentheticals or asides), and has gendered associations with prosodic markers and interactional discourse particles.Keywords
This publication has 62 references indexed in Scilit:
- Temporal, structural, and pragmatic synchrony between intonation and gestureLaboratory Phonology, 2012
- Localizing the Global on the Participatory Web2010
- Coding gestural behavior with the NEUROGES-ELAN systemBehavior Research Methods, 2009
- ‘So let's talk. Let's chat. Let's start a dialog’: An analysis of the conversation metaphor employed in Clinton's and Obama's YouTube campaign clipsMultilingua, 2008
- Folk notions of um and uh, you know, and likeText & Talk, 2007
- Vocal communication of emotion: A review of research paradigmsSpeech Communication, 2003
- The portrayal of women’s images in magazine advertisements: Goffman’s gender analysis revisitedSex Roles, 1997
- Do Gestures Communicate? A ReviewResearch on Language and Social Interaction, 1994
- Strangers meet: Laughter and nonverbal signs of interest in opposite-sex encountersJournal of Nonverbal Behavior, 1990
- On the Interpretation of χ 2 from Contingency Tables, and the Calculation of PJournal of the Royal Statistical Society, 1922