Advances in phonetics-based sub-unit modeling for transcription alignment and sign language recognition
Authors:
- 1 June 2011
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21607508,p. 1-6
Abstract
We explore novel directions for incorporating phonetic transcriptions into sub-unit based statistical models for sign language recognition. First, we employ a new symbolic processing approach for converting sign language annotations, based on HamNoSys symbols, into structured sequences of labels according to the Posture-Detention-Transition-Steady Shift phonetic model. Next, we exploit these labels, and their correspondence with visual features to construct phonetics-based statistical sub-unit models. We also align these sequences, via the statistical sub-unit construction and decoding, to the visual data to extract time boundary information that they would lack otherwise. The resulting phonetic sub-units offer new perspectives for sign language analysis, phonetic modeling, and automatic recognition. We evaluate this approach via sign language recognition experiments on an extended Lemmas Corpus of Greek Sign Language, which results not only in improved performance compared to pure data-driven approaches, but also in meaningful phonetic sub-unit models that can be further exploited in interdisciplinary sign language analysis.Keywords
This publication has 6 references indexed in Scilit:
- A Segmental Framework for Representing Signs PhoneticallySign Language Studies, 2011
- Modelling and recognition of the linguistic components in American Sign LanguageImage and Vision Computing, 2009
- Modelling and segmenting subunits for sign language recognition based on hand motion analysisPattern Recognition Letters, 2009
- Learning the basic units in American Sign Language using discriminative segmental feature selection2009
- A Framework for Recognizing the Simultaneous Aspects of American Sign LanguageComputer Vision and Image Understanding, 2001
- American Sign Language: The Phonological BaseSign Language Studies, 1989