Proceedings ArticleDOI
Continuous Chinese sign language recognition with CNN-LSTM
TLDR
An appropriate model based on convolutional neural network (CNN) combined with Long Short-Term Memory (LSTM) network is formulated in order to accomplish the continuous recognition work of real-time SLR system.Abstract:
The goal of sign language recognition (SLR) is to translate the sign language into text, and provide a convenient tool for the communication between the deaf-mute and the ordinary. In this paper, we formulate an appropriate model based on convolutional neural network (CNN) combined with Long Short-Term Memory (LSTM) network, in order to accomplish the continuous recognition work. With the strong ability of CNN, the information of pictures captured from Chinese sign language (CSL) videos can be learned and transformed into vector. Since the video can be regarded as an ordered sequence of frames, LSTM model is employed to connect with the fully-connected layer of CNN. As a recurrent neural network (RNN), it is suitable for sequence learning tasks with the capability of recognizing patterns defined by temporal distance. Compared with traditional RNN, LSTM has performed better on storing and accessing information. We evaluate this method on our self-built dataset including 40 daily vocabularies. The experimental results show that the recognition method with CNN-LSTM can achieve a high recognition rate with small training sets, which will meet the needs of real-time SLR system.read more
Citations
More filters
Journal ArticleDOI
Arabic Sign Language Recognition through Deep Neural Networks Fine-Tuning
Yaser Saleh,Ghassan Issa +1 more
TL;DR: Transfer learning and fine tuning deep convolutional neural networks are utilized to improve the accuracy of recognizing 32 hand gestures from the Arabic sign language.
Journal ArticleDOI
Technical Approaches to Chinese Sign Language Processing: A Review
TL;DR: This survey provides an overview of the most important work on Chinese sign language recognition and translation, discussed its classification, highlights the features explored in sign language Recognition research, presents the datasets available, and provides trends for the future research.
Journal ArticleDOI
Convolutional and recurrent neural network for human activity recognition: Application on American sign language.
TL;DR: This study proposes to classify 60 signs from the American Sign Language based on data provided by the LeapMotion sensor by using different conventional machine learning and deep learning models including a model called DeepConvLSTM that integrates convolutional and recurrent layers with Long-Short Term Memory cells.
Journal ArticleDOI
An Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences
TL;DR: This work replaces the traditional encoder in a neural machine translation (NMT) module with an improved architecture, which incorporates a temporal convolution (T-Conv) unit and a dynamic hierarchical bidirectional GRU (DH-BiGRU) unit sequentially.
Journal ArticleDOI
Multimodal Spatiotemporal Networks for Sign Language Recognition
TL;DR: A multimodal deep learning architecture for sign language recognition which effectively combines RGB-D input and two-stream spatiotemporal networks is proposed which obtains the state-the-of-art performance on the datasets of CSL and IsoGD.
References
More filters
Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
Journal ArticleDOI
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Posted Content
ADADELTA: An Adaptive Learning Rate Method
TL;DR: A novel per-dimension learning rate method for gradient descent called ADADELTA that dynamically adapts over time using only first order information and has minimal computational overhead beyond vanilla stochastic gradient descent is presented.
Book
Supervised Sequence Labelling with Recurrent Neural Networks
TL;DR: A new type of output layer that allows recurrent networks to be trained directly for sequence labelling tasks where the alignment between the inputs and the labels is unknown, and an extension of the long short-term memory network architecture to multidimensional data, such as images and video sequences.
Dissertation
Visual Recognition of American Sign Language Using Hidden Markov Models.
TL;DR: Using hidden Markov models (HMM's), an unobstrusive single view camera system is developed that can recognize hand gestures, namely, a subset of American Sign Language (ASL), achieving high recognition rates for full sentence ASL using only visual cues.