A Framework for Recognizing the Simultaneous Aspects of American Sign Language

TLDR: This paper presents a novel framework to ASL recognition that aspires to being a solution to the scalability problems, based on breaking down the signs into their phonemes and modeling them with parallel hidden Markov models.

About: This article is published in Computer Vision and Image Understanding.The article was published on 2001-03-01 and is currently open access. It has received 321 citations till now. The article focuses on the topics: Hidden Markov model & Gesture recognition.

Figures

TABLE 1 Partial List of Movements

FIG. 2. The signs for “interpreter” (top) and “teacher” (bottom) illustrate sequential contrast. They differ only in the first part of their movement sequence (left). The movements that make up this sign are examples of phonemes.

FIG. 9. The sign for “inform” demonstrates how several features in ASL change simultaneously. Both hands move, starting at different body locations. The handshape is symmetrical, but changes from a closed fist to a half-open hand during the sign.

FIG. 11. Movement epenthesis. The arrow in the middle picture indicates an extra movement between the signs for “father” and “read” that is not present in their lexical forms.

FIG. 10. The sign for “father,” where the different features are modeled in separate channels. The handshape and orientations stay the same during the entire sign, so only one phoneme appears in each of these channels. Compare with Fig. 8.

TABLE 2 Regular HMMs: Results of the Recognition Experiments