On the robustness of audiovisual liveness detection to visual speech animation
- 1 September 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
Audiovisual speech synchrony detection is an important liveness check for talking face verification systems to make sure that the (pre-defined) content and timing of the given audible and visual speech samples match. Nowadays, there exists virtually no technical limitations for combining transferable facial animation and voice conversion (or synthesis) to create an ultimate audiovisual artifact that is able to spoof even advanced random challenge-response based liveness detection. In this study, we investigate the performance of the state-of-the-art text-independent lip-sync detection techniques under presentation attacks consisting of audio recordings of the targeted person and corresponding animated visual speech. Our experimental analysis with three different photo-realistic visual speech animation techniques reveals that generic synchrony models can be fooled even with underarticulated but synchronized lip movements. Thus, measuring audio-video synchrony or content alone is not enough for securing audiovisual biometric systems. Our preliminary findings suggest though that adaptation of person-specific audiovisual speech dynamics is one possible approach to tackle these kinds of high-effort attacks.Keywords
This publication has 17 references indexed in Scilit:
- Audiovisual speech synthesis: An overview of the state-of-the-artSpeech Communication, 2015
- One millisecond face alignment with an ensemble of regression trees2014
- Motion recognition using local auto-correlation of space–time gradientsPattern Recognition Letters, 2012
- Talking-Face Identity Verification, Audiovisual Forgery, and Robustness IssuesEURASIP Journal on Advances in Signal Processing, 2009
- Audio-visual speech asynchrony detection using co-inertia analysis and coupled hidden markov modelsPattern Analysis and Applications, 2008
- Making talking-face authentication robust to deliberate impostureInternational Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2008
- Local spatiotemporal descriptors for visual recognition of spoken phrases2007
- Transferable videorealistic speech animation2005
- Trainable videorealistic speech animation2002
- Co‐inertia analysis: an alternative method for studying species–environment relationshipsFreshwater Biology, 1994