Speaking with hands
Authors:
- Matthew StoneMSMatthew Stone
- Alyssa LeesALAlyssa Lees
- Chris BreglerCBChris Bregler
Open Access
- 1 August 2004
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 23 (3) , 506-513
Abstract
We describe a method for using a database of recorded speech and captured motion to create an animated conversational character. People's utterances are composed of short, clearly-delimited phrases; in each phrase, gesture and speech go together meaningfully and synchronize at a common point of maximum emphasis. We develop tools for collecting and managing performance data that exploit this structure. The tools help create scripts for performers, help annotate and segment performance data, and structure specific messages for characters to use within application contexts. Our animations then reproduce this structure. They recombine motion samples with new speech samples to recreate coherent phrases, and blend segments of speech and motion together phrase-by-phrase into extended utterances. By framing problems for utterance generation and synthesis so that they can draw closely on a talented performance, our techniques support the rapid construction of animated characters with rich and appropriate expression.Keywords
This publication has 39 references indexed in Scilit:
- Motion synthesis from annotationsACM Transactions on Graphics, 2003
- Rhythmic-motion synthesis based on motion-beat analysisACM Transactions on Graphics, 2003
- Interactive control of avatars animated with human motion dataACM Transactions on Graphics, 2002
- Eyes aliveACM Transactions on Graphics, 2002
- Motion textureACM Transactions on Graphics, 2002
- Visible Acts of MeaningJournal of Language and Social Psychology, 2000
- Embodied conversational interface agentsCommunications of the ACM, 2000
- Verbs and adverbs: multidimensional motion interpolationIEEE Computer Graphics and Applications, 1998
- Generating Facial Expressions for SpeechCognitive Science, 1996
- Performance-driven facial animationACM SIGGRAPH Computer Graphics, 1990