Evaluation of a Korean Lip-sync system for an android robot
Authors:
- 7 November 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 78-82
Abstract
Lip-syncing of android robots resembling people is essential to accurately convey their intentions to humans. In this paper, we develop a system of Korean lip-syncing, with the assumption that people can guess a word or phrase from watching a lip-syncing robot without sound. The mouth shape for 10 single vowels was generated based on a Korean single vowels triangle chart. Robots can lip-sync in real time a variety of words and sentences using 10 mouth shapes. We performed experiments recording a mouth robot and an announcer reading text. We conducted a survey to assess humans guessing the representations of a female announcer and of a robot to compare the percent of correct answers in each case. Additionally, we also conducted a survey of robot mouth shapes and lip-sync timing to assess the reaction of subjects on 5-Likert scales. Results indicate that the percent of correct guesses from the mouth shape of the robot was one third of that from the human announcer. Subjects assessed the mouth shape and lip-sync timing of the robot as being somewhat unnatural. We expect that android robot lip-syncing currently uses mouth shapes that are perceived as lying in the uncanny valley when subjects try to interpret them. Thus, we will present a more natural mouth shape, add mouth shapes for diphthongs, and develop a mouth shape that varies with voice volume, improving the rate of lip-sync recognition.Keywords
This publication has 9 references indexed in Scilit:
- Analysis of android robot lip-sync factors affecting communication2016
- Korean speech recognition using phonemics for lip-sync animation2014
- The Uncanny Valley [From the Field]IEEE Robotics & Automation Magazine, 2012
- Animating Lip-Sync Characters With Dominated Animeme ModelsIEEE Transactions on Circuits and Systems for Video Technology, 2012
- VocaWatcher: Natural singing motion generator for a humanoid robot2011
- Real-time lip synchronization between text-to-speech (TTS) system and robot mouth2010
- Emotive qualities in lip-synchronized robot speechAdvanced Robotics, 2003
- Hearing lips and seeing voicesNature, 1976
- Confusions Among Visually Perceived ConsonantsJournal of Speech and Hearing Research, 1968