Microsoft Kinect, a low-cost motion sensing device, enables users to interact with computers or game consoles naturally through gestures and spoken commands without any other peripheral equipment. As such, it has commanded intense interests in research and development on the Kinect technology. In this paper, we present, a comprehensive survey on Kinect applications, and the latest research and development on motion recognition using data captured by the Kinect sensor. On the applications front, we review the applications of the Kinect technology in a variety of areas, including healthcare, education and performing arts, robotics, sign language recognition, retail services, workplace safety training, as well as 3D reconstructions. On the technology front, we provide an overview of the main features of both versions of the Kinect sensor together with the depth sensing technologies used, and review literatures on human motion recognition techniques used in Kinect applications. We provide a classification of motion recognition techniques to highlight the different approaches used in human motion recognition. Furthermore, we compile a list of publicly available Kinect datasets. These datasets are valuable resources for researchers to investigate better methods for human motion recognition and lower-level computer vision tasks such as segmentation, object detection and human pose estimation.

/pdf/a-survey-of-applications-and-human-motion-recognition-with-hags1y076t.pdf

A Survey of Applications and Human Motion Recognition with Microsoft Kinect

Sign language facilitates communication between hearing impaired peoples and the rest of the society. A number of sign language recognition (SLR) systems have been developed by researchers, but they are limited to isolated sign gestures only. In this paper, we propose a modified long short-term memory (LSTM) model for continuous sequences of gestures or continuous SLR that recognizes a sequence of connected gestures. It is based on splitting of continuous signs into sub-units and modeling them with neural networks. Thus, the consideration of a different combination of sub-units is not required during training. The proposed system has been tested with 942 signed sentences of Indian Sign Language (ISL). These sign sentences are recognized using 35 different sign words. The average accuracy of 72.3% and 89.5% has been recorded on signed sentences and isolated sign words, respectively.

A Modified LSTM Model for Continuous Sign Language Recognition Using Leap Motion

Despite the importance of sign language recognition systems, there is a lack of a Systematic Literature Review and a classification scheme for it. This is the first identifiable academic literature review of sign language recognition systems. It provides an academic database of literature between the duration of 2007–2017 and proposes a classification scheme to classify the research articles. Three hundred and ninety six research articles were identified and reviewed for their direct relevance to sign language recognition systems. One hundred and seventeen research articles were subsequently selected, reviewed and classified. Each of 117 selected papers was categorized on the basis of twenty five sign languages and were further compared on the basis of six dimensions (data acquisition techniques, static/dynamic signs, signing mode, single/double handed signs, classification technique and recognition rate). The Systematic Literature Review and classification process was verified independently. Literature findings of this paper indicate that the major research on sign language recognition has been performed on static, isolated and single handed signs using camera. Overall, it will be hoped that the study may provide readers and researchers a roadmap to guide future research and facilitate knowledge accumulation and creation into the field of sign language recognition.

Sign Language Recognition Systems: A Decade Systematic Literature Review

Selfie video based continuous Indian sign language recognition system

Structured dynamic time warping for continuous hand trajectory gesture recognition

Continuous sign language recognition using level building based on fast hidden Markov model

Isolated sign language recognition (SLR) is a long-standing research problem. The existing methods consider inclusively ambiguous data to represent a sign and ignore the fact that only scarce key information can represent the sign efficiently since most information are redundant. Furthermore, inclusion of redundant information may result in inefficiency and difficulty in modeling the long-term dependency for SLR. This letter delivers a novel sequence-to-sequence learning method based on keyframe centered clips (KCCs) for Chinese SLR. Different from conventional methods, only key information is considered to represent a sign significantly. The frames-to-word task is transformed into a KCCs-to-subwords task successfully, to allow for different attention in the input data. The empirical results of the proposed method outperform significantly the state-of-the-art SLR systems on our dataset containing 310 Chinese sign language words.

A Novel Chinese Sign Language Recognition Method Based on Keyframe-Centered Clips

An isolated sign language recognition system is presented in this paper. A RGB-D sensor, Microsoft Kinect, is used for obtaining color stream and skeleton points from the depth stream. For a particular sign we extract a representative feature vector composed by hand trajectories and hand shapes. A sparse dictionary learning algorithm, Label Consistent K-SVD (LC-KSVD), is applied to obtain a discriminative dictionary. Based on that, we further develop a new classification approach to get better result. Our system is evaluated on 34 isolated Chinese sign words including one-handed signs and two-handed signs. Experimental results show the proposed system gets high recognition accuracy, of the reported 96.75%, and obtain an average accuracy of 92.36% for signer independent recognition.

An Isolated Sign Language Recognition System Using RGB-D Sensor with Sparse Coding

Sign language recognition (SLR) plays an important role in communication between deaf and hearing society. However, the recognition result is still worse for signer independent recognition. The reason is that there exists large variation between the signs from different subjects. In this paper, weighted hidden markov model (HMM) is proposed to deal with the variation. Unlike traditional HMM, WHMM assigns each sign samples with different weights. For the sign sample with big variation, the sample weight is big accordingly. Furthermore, we utilize Kinect to produce robust sign features to improve recognition rate. Our system is evaluated on one Chinese sign language dataset of 156 isolated sign words. Experimental result shows our proposed method outperforms other methods with a high recognition rate of 94.74%.

Sign Language Recognition System Based on Weighted Hidden Markov Model

Background In order to overcome the limitation of long scanning time, compressive sensing (CS) technology exploits the sparsity of image in some transform domain to reduce the amount of acquired data. Therefore, CS has been widely used in magnetic resonance imaging (MRI) reconstruction. Discussion Blind compressed sensing enables to recover the image successfully from highly under- sampled measurements, because of the data-driven adaption of the unknown transform basis priori. Moreover, analysis-based blind compressed sensing often leads to more efficient signal reconstruction with less time than synthesis-based blind compressed sensing. Recently, some experiments have shown that nonlocal low-rank property has the ability to preserve the details of the image for MRI reconstruction. Methods Here, we focus on analysis-based blind compressed sensing, and combine it with additional nonlocal low-rank constraint to achieve better MR images from fewer measurements. Instead of nuclear norm, we exploit non-convex Schatten p-functionals for the rank approximation. Results & conclusion Simulation results indicate that the proposed approach performs better than the previous state-of-the-art algorithms.

Jinxu Tao

Papers

Continuous sign language recognition using level building based on fast hidden Markov model

A Novel Chinese Sign Language Recognition Method Based on Keyframe-Centered Clips

An Isolated Sign Language Recognition System Using RGB-D Sensor with Sparse Coding

Sign Language Recognition System Based on Weighted Hidden Markov Model

An Algorithm Combining Analysis-based Blind Compressed Sensing and Nonlocal Low-rank Constraints for MRI Reconstruction.