Sign Language Spotting with a Threshold Model Based on Conditional Random Fields
Authors:
Top Cited Papers
- 27 June 2008
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 31 (7) , 1264-1277
Abstract
Sign language spotting is the task of detecting and recognizing signs in a signed utterance, in a set vocabulary. The difficulty of sign language spotting is that instances of signs vary in both motion and appearance. Moreover, signs appear within a continuous gesture stream, interspersed with transitional movements between signs in a vocabulary and nonsign patterns (which include out-of-vocabulary signs, epentheses, and other movements that do not correspond to signs). In this paper, a novel method for designing threshold models in a conditional random field (CRF) model is proposed which performs an adaptive threshold for distinguishing between signs in a vocabulary and nonsign patterns. A short-sign detector, a hand appearance-based sign verification method, and a subsign reasoning method are included to further improve sign language spotting accuracy. Experiments demonstrate that our system can spot signs from continuous data with an 87.0 percent spotting rate and can recognize signs from isolated data with a 93.5 percent recognition rate versus 73.5 percent and 85.4 percent, respectively, for CRFs without a threshold model, short-sign detection, subsign reasoning, and hand appearance-based sign verification. Our system can also achieve a 15.0 percent sign error rate (SER) from continuous data and a 6.4 percent SER from isolated data versus 76.2 percent and 14.5 percent, respectively, for conventional CRFs.Keywords
This publication has 30 references indexed in Scilit:
- Aligning ASL for Statistical Translation Using a Discriminative Word Model2006
- MULTIPLE HUMAN DETECTION AND TRACKING BASED ON WEIGHTED TEMPORAL TEXTURE FEATURESInternational Journal of Pattern Recognition and Artificial Intelligence, 2006
- Histograms of Oriented Gradients for Human Detection2005
- Simultaneous Localization and Recognition of Dynamic Hand Gestures2005
- Face Detection and Facial Component Extraction by Wavelet Decomposition and Support Vector Machines2003
- Extraction of 2D motion trajectories and its application to hand gesture recognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Machine Learning for Sequential Data: A Review2002
- Video-based sign recognition using self-organizing subunits2002
- A Framework for Recognizing the Simultaneous Aspects of American Sign LanguageComputer Vision and Image Understanding, 2001
- Training and search algorithms for an interactive wordspotting system1992