Ala addin I. Sidig

Bio: Ala addin I. Sidig is an academic researcher from King Fahd University of Petroleum and Minerals. The author has contributed to research in topics: Sign language & Gesture. The author has an hindex of 3, co-authored 10 publications receiving 48 citations.

KArSL: Arabic Sign Language Database

Abstract: Sign language is the major means of communication for the deaf community. It uses body language and gestures such as hand shapes, lib patterns, and facial expressions to convey a message. Sign language is geography-specific, as it differs from one country to another. Arabic Sign language is used in all Arab countries. The availability of a comprehensive benchmarking database for ArSL is one of the challenges of the automatic recognition of Arabic Sign language. This article introduces KArSL database for ArSL, consisting of 502 signs that cover 11 chapters of ArSL dictionary. Signs in KArSL database are performed by three professional signers, and each sign is repeated 50 times by each signer. The database is recorded using state-of-art multi-modal Microsoft Kinect V2. We also propose three approaches for sign language recognition using this database. The proposed systems are Hidden Markov Models, deep learning images’ classification model applied on an image composed of shots of the video of the sign, and attention-based deep learning captioning system. Recognition accuracies of these systems indicate their suitability for such a large number of Arabic signs. The techniques are also tested on a publicly available database. KArSL database will be made freely available for interested researchers.

Automated sign language recognition

Abstract: A sign language recognizer is configured to detect interest points in an extracted sign language feature, wherein the interest points are localized in space and time in each image acquired from a plurality of frames of a sign language video; apply a filter to determine one or more extrema of a central region of the interest points; associate features with each interest point using a neighboring pixel function; cluster a group of extracted sign language features from the images based on a similarity between the extracted sign language features; represent each image by a histogram of visual words corresponding to the respective image to generate a code book; train a classifier to classify each extracted sign language feature using the code book; detect a posture in each frame of the sign language video using the trained classifier; and construct a sign gesture based on the detected postures.

Arabic Sign Language Recognition Using Optical Flow-Based Features and HMM

Abstract: Sign language is the main communication channel of deaf community It uses gestures and body language such as facial expressions, lib patterns, and hand shapes to convey meaning Sign language differs from one country to another Sign language recognition helps in removing barriers between people who understand only spoken language and those who understand sign language In this work, we propose an algorithm for segmenting videos of signs into sequences of still images and four techniques for Arabic sign language recognition, namely Modified Fourier Transform (MFT), Local Binary Pattern (LBP), Histogram of Oriented Gradients (HOG), and combination of HOG and Histogram of Optical Flow (HOG-HOF) These techniques are evaluated using Hidden Markov Model (HMM) The best performance is obtained with MFT features with 9911% accuracy This recognition rate shows the correctness and robustness of the proposed signs’ video segmentation algorithm

Trajectory based Arabic Sign Language Recognition

Abstract: Deaf and hearing impaired people use their hand as a tongue to convey their thoughts by performing descriptive gestures that form the sign language. A sign language recognition system is a system that translates these gestures into a form of spoken language. Such systems are faced by several challenges, like the high similarities of the different signs, difficulty in determining the start and end of signs, lack of comprehensive and bench marking databases. This paper proposes a system for recognition of Arabic sign language using the 3D trajectory of hands. The proposed system models the trajectory as a polygon and finds features that describes this polygon and feed them to a classifier to recognize the signed word. The system is tested on a database of 100 words collected using Kinect. The work is compared with other published works using publicly available dataset which reflects the superiority of the proposed technique. The system is tested for both signer-dependent and signer-independent recognition.

A Review on Systems-Based Sensory Gloves for Sign Language Recognition State of the Art between 2007 and 2017.

Abstract: Loss of the ability to speak or hear exerts psychological and social impacts on the affected persons due to the lack of proper communication. Multiple and systematic scholarly interventions that vary according to context have been implemented to overcome disability-related difficulties. Sign language recognition (SLR) systems based on sensory gloves are significant innovations that aim to procure data on the shape or movement of the human hand. Innovative technology for this matter is mainly restricted and dispersed. The available trends and gaps should be explored in this research approach to provide valuable insights into technological environments. Thus, a review is conducted to create a coherent taxonomy to describe the latest research divided into four main categories: development, framework, other hand gesture recognition, and reviews and surveys. Then, we conduct analyses of the glove systems for SLR device characteristics, develop a roadmap for technology evolution, discuss its limitations, and provide valuable insights into technological environments. This will help researchers to understand the current options and gaps in this area, thus contributing to this line of research.

DeepArSLR: A Novel Signer-Independent Deep Learning Framework for Isolated Arabic Sign Language Gestures Recognition

Abstract: Hand gesture recognition has attracted the attention of many researchers due to its wide applications in robotics, games, virtual reality, sign language and human-computer interaction. Sign language is a structured form of hand gestures and the most effective communication way among hear-impaired people. Developing an efficient sign language recognition system to recognize dynamic isolated gestures encounters three major challenges, namely, hand segmentation, hand shape feature representation and gesture sequence recognition. Traditional sign language recognition methods utilize color-based hand segmentation algorithms to segment hands, hand-crafted feature extraction for hand shape representation and Hidden Markov Model (HMM) for sequence recognition. In this paper, a novel framework is proposed for signer-independent sign language recognition using multiple deep learning architectures comprising hand semantic segmentation, hand shape feature representation and deep recurrent neural network. The recently developed semantic segmentation method called DeepLabv3+ is trained using a set of pixel-labeled hand images to extract hand regions from each frame of the input video. Then, the extracted hand regions are cropped and scaled to a fixed size to alleviate hand scale variations. Extracting hand shape features is achieved using a single layer Convolutional Self-Organizing Map (CSOM) instead of relying on transfer learning of pre-trained deep convolutional neural networks. The sequence of extracted feature vectors are then recognized using deep Bi-directional Long Short-Term Memory (BiLSTM) recurrent neural network. BiLSTM network contains three BiLSTM layers, one fully connected and softmax layers. The performance of the proposed method is evaluated using a challenging Arabic sign language database containing 23 isolated words captured from three different users. Experimental results show that the performance of proposed framework outperforms with large margin the state-of-the-art methods for signer-independent testing strategy.

An Intelligent Arabic Sign Language Recognition System Using a Pair of LMCs With GMM Based Classification

Abstract: In this paper, we propose a dual leap motion controllers (LMC)-based Arabic sign language recognition system. More specifically, we introduce the concept of using both front and side LMCs to cater for the challenges of finger occlusions and missing data. For feature extraction, an optimum set of geometric features is selected from both controllers, while for classification, we used both a Bayesian approach with a Gaussian mixture model (GMM) and a simple linear discriminant analysis (LDA) approach. To combine the information from the two LMCs, we introduce an evidence-based fusion approach; namely, the Dempster–Shafer (DS) theory of evidence. Data were collected from two native adult signers, for 100 isolated Arabic dynamic signs. Ten observations were collected for each of the signs. The proposed framework uses an intelligent strategy to handle the case of missing data from one or both controllers. A recognition accuracy of about 92% was achieved. The proposed system outperforms state-of-the-art glove-based systems and single-sensor-based techniques.

A Novel Natural Language Processing (NLP)–Based Machine Translation Model for English to Pakistan Sign Language Translation

Abstract: The deaf community in the world uses a gesture-based language, generally known as sign language. Every country has a different sign language; for instance, USA has American Sign Language (ASL) and UK has British Sign Language (BSL). The deaf community in Pakistan uses Pakistan Sign Language (PSL), which like other natural languages, has a vocabulary, sentence structure, and word order. Majority of the hearing community is not aware of PSL due to which there exists a huge communication gap between the two groups. Similarly, deaf persons are unable to read text written in English and Urdu. Hence, the provision of an effective translation model can support the cognitive capability of the deaf community to interpret natural language materials available on the Internet and in other useful resources. This research involves exploiting natural language processing (NLP) techniques to support the deaf community by proposing a novel machine translation model that translates English sentences into equivalent Pakistan Sign Language (PSL). Though a large number of machine translation systems have been successfully implemented for natural to natural language translations, natural to sign language machine translation is a relatively new area of research. State-of-the-art works in natural to sign language translation are mostly domain specific and suffer from low accuracy scores. Major reasons are specialised language structures for sign languages, and lack of annotated corpora to facilitate development of more generalisable machine translation systems. To this end, a grammar-based machine translation model is proposed to translate sentences written in English language into equivalent PSL sentences. To the best of our knowledge, this is a first effort to translate any natural language to PSL using core NLP techniques. The proposed approach involves a structured process to investigate the linguistic structure of PSL and formulate the grammatical structure of PSL sentences. These rules are then formalised into a context-free grammar, which, in turn, can be efficiently implemented as a parsing module for translation and validation of target PSL sentences. The whole concept is implemented as a software system, comprising the NLP pipeline and an external service to render the avatar-based video of translated words, in order to compensate the cognitive hearing deficit of deaf people. The accuracy of the proposed translation model has been evaluated manually and automatically. Quantitative results reveal a very promising Bilingual Evaluation Understudy (BLEU) score of 0.78. Subjective evaluations demonstrate that the system can compensate for the cognitive hearing deficit of end users through the system output expressed as a readily interpretable avatar. Comparative analysis shows that our proposed system works well for simple sentences but struggles to translate compound and compound complex sentences correctly, which warrants future ongoing research.

Automatic translation of Arabic text-to-Arabic sign language

Abstract: Arabic sign language (ArSL) is a full natural language that is used by the deaf in Arab countries to communicate in their community. Unfamiliarity with this language increases the isolation of deaf people from society. This language has a different structure, word order, and lexicon than Arabic. The translation between ArSL and Arabic is a complete machine translation challenge, because the two languages have different structures and grammars. In this work, we propose a rule-based machine translation system to translate Arabic text into ArSL. The proposed system performs a morphological, syntactic, and semantic analysis on an Arabic sentence to translate it into a sentence with the grammar and structure of ArSL. To transcribe ArSL, we propose a gloss system that can be used to represent ArSL. In addition, we develop a parallel corpus in the health domain, which consists of 600 sentences, and will be freely available for researchers. We evaluate our translation system on this corpus and find that our translation system provides an accurate translation for more than 80% of the translated sentences.