Recent methods and databases in vision-based hand gesture recognition

In this article, we introduce the application of rigorous analysis procedures to goal models to provide several benefits beyond the initial act of modeling. Such analysis can allow modelers to assess the satisfaction of goals, facilitate evaluation of high-level design alternatives, help analysts decide on the high-level requirements and design of the system, test the sanity of a model, and support communication and learning. The analysis of goal models can be done in very different ways depending on the nature of the model and the purpose of the analysis. In our work, we use the Goal-oriented Requirement Language (GRL), which is part of the User Requirements Notation (URN). URN, a new Recommendation of the International Telecommunications Union, provides the first standard goal-oriented language. Using GRL, we develop an approach to analysis that can be done by evaluating qualitative or quantitative satisfaction levels of the actors and intentional elements (e.g., goals and tasks) composing the model. Initial satisfaction levels for some of the intentional elements are provided in a strategy and then propagated to the other intentional elements of the model through the various links that connect them. The results allow for an assessment of the relative effectiveness of design alternatives at the requirements level. Although no specific propagation algorithm is imposed in the URN standard, different criteria for defining evaluation mechanisms are described. We provide three algorithms (quantitative, qualitative, and hybrid) as examples, which satisfy the constraints imposed by the standard. These algorithms have been implemented in the open-source jUCMNav tool, an Eclipse-based editor for URN models. The algorithms are presented and compared with the help of a telecommunication system example. © 2010 Wiley Periodicals, Inc.

http://cserg.site.uottawa.ca/ucm/pub/UCM/VirLibIJIS09/IJIS2010-Web.pdf

Evaluating goal models within the goal-oriented requirement language

This paper discusses the development of a natural gesture user interface that tracks and recognizes in real time hand gestures based on depth data collected by a Kinect sensor. The interest space corresponding to the hands is first segmented based on the assumption that the hand of the user is the closest object in the scene to the camera. A novel algorithm is proposed to improve the scanning time in order to identify the first pixel on the hand contour within this space. Starting from this pixel, a directional search algorithm allows for the identification of the entire hand contour. The $k$ -curvature algorithm is then employed to locate the fingertips over the contour, and dynamic time warping is used to select gesture candidates and also to recognize gestures by comparing an observed gesture with a series of prerecorded reference gestures. The comparison of results with state-of-the-art approaches shows that the proposed system outperforms most of the solutions for the static recognition of sign digits and is similar in terms of performance for the static and dynamic recognition of popular signs and for the sign language alphabet. The solution simultaneously deals with static and dynamic gestures as well as with multiple hands within the interest space. An average recognition rate of 92.4% is achieved over 55 static and dynamic gestures. Two possible applications of this work are discussed and evaluated: one for interpretation of sign digits and gestures for a friendlier human-machine interaction and the other one for the natural control of a software interface.

Static and Dynamic Hand Gesture Recognition in Depth Data Using Dynamic Time Warping

The User Requirements Notation (URN), standardized by the International Telecommunication Union in 2008, is used to model and analyze requirements with goals and scenarios. This paper describes the first ten years of development of URN, and discusses ongoing efforts targeting the next ten years. We did a study inspired by the systematic literature review approach, querying five major search engines and using the existing URN Virtual Library. Based on the 281 scientific publications related to URN we collected and analyzed, we observe a shift from a more conventional use of URN for telecommunications and reactive systems to business process management and aspect-oriented modeling, with relevant extensions to the language being proposed. URN also benefits from a global and active research community, although industrial contributions are still sparse. URN is now a leading language for goal-driven and scenario-oriented modeling with a promising future for many application domains.

User Requirements Notation: The First Ten Years, The Next Ten Years (Invited Paper)

In this paper, we present an accelerometer-based pen-type sensing device and a user-independent hand gesture recognition algorithm. Users can hold the device to perform hand gestures with their preferred handheld styles. Gestures in our system are divided into two types: the basic gesture and the complex gesture, which can be represented as a basic gesture sequence. A dictionary of 24 gestures, including 8 basic gestures and 16 complex gestures, is defined. An effective segmentation algorithm is developed to identify individual basic gesture motion intervals automatically. Through segmentation, each complex gesture is segmented into several basic gestures. Based on the kinematics characteristics of the basic gesture, 25 features are extracted to train the feedforward neural network model. For basic gesture recognition, the input gestures are classified directly by the feedforward neural network classifier. Nevertheless, the input complex gestures go through an additional similarity matching procedure to identify the most similar sequences. The proposed recognition algorithm achieves almost perfect user-dependent and user-independent recognition accuracies for both basic and complex gestures. Experimental results based on 5 subjects, totaling 1600 trajectories, have successfully validated the effectiveness of the feedforward neural network and similarity matching-based gesture recognition algorithm.

Accelerometer-Based Hand Gesture Recognition by Neural Network and Similarity Matching

This paper presents the state-of-the art dynamic sign language recognition (DSLR) system for smart home interactive applications Our novel DSLR system comprises two main subsystems: an image processing (IP) module and a stochastic linear formal grammar (SLFG) module Our IP module enables us to recognize the individual words of the sign language (ie, a single gesture) In this module, we used the bag-of-features (BOFs) and a local part model approach for bare hand dynamic gesture recognition from a video We used dense sampling to extract local 3-D multiscale whole-part features We adopted 3-D histograms of a gradient orientation descriptor to represent features The $k$ -means++ method was applied to cluster the visual words Dynamic hand gesture classification was conducted using the BOFs and nonlinear support vector machine methods We used a multiscale local part model to preserve temporal context The SLFG module analyzes the sentences of the sign language (ie, sequences of gestures) and determines whether or not they are syntactically valid Therefore, the DSLR system is not only able to rule out ungrammatical sentences, but it can also make predictions about missing gestures, which, in turn, increases the accuracy of our recognition task Our IP module alone seals the accuracy of 97% and outperforms any existing bare hand dynamic gesture recognition system However, by exploiting syntactic pattern recognition, the SLFG module raises this accuracy by 165% This makes the aggregate performance of the DSLR system as accurate as 9865%

Dynamic Sign Language Recognition for Smart Home Interactive Application Using Stochastic Linear Formal Grammar

Digital signage is a very attractive medium for advertisement and general communications in public open spaces. In order to add interaction capabilities to digital signage displays, special considerations must be taken. For example, the signs' environment and placement might prevent direct access to conventional means of interaction, such as using a keyboard or a touch-sensitive screen. This paper describes a vision-based gesture recognition approach to interact with digital signage systems and discusses the issues faced by such systems. Using Haar-like features and the AdaBoosting algorithm, a set of hand gestures can be recognized in real-time and converted to gesture commands to control and manipulate the digital signage display. A demonstrative application using this gesture recognition interface is also depicted.

Interacting with Digital Signage Using Hand Gestures

This paper discusses inter-robot and human-robot communication by bare hand dynamic gestures. We use a Bag-of-Features and a local part model approach for bare hand dynamic hand gesture recognition from video. We used dense sampling to extract local 3D multiscale whole-part features. We adopted three dimensional histograms of a gradient orientation (3D HOG) descriptor to represent features. The K-means++ method was applied to cluster the visual words. Dynamic hand gesture classification was completed by using a Bag-of-features (BOF) and non-linear support vector machine (SVM) method. A BOF does not track the order of events. To counter the unordered events of the BOF approach, we used a multiscale local part model to preserve temporal context. Initial experimental results on the newly collected complex dataset show a higher level of recognition. We used the same above mentioned approach for inter-robot communication by using two sample hand models.

Dynamic hand gesture recognition for human-robot and inter-robot communication

This paper presents a system for recognition of dynamic sign language and voice recognition for smart home interactive applications. We use the Bag-of-Features and a local part model approach for bare hand dynamic gesture recognition from video. We use a dense sampling to extract local 3D multiscale whole-part features. We adopted three-dimensional histograms of a gradient orientation (3D HOG) descriptor to represent features. The K-means++ method was applied to cluster the visual words. Dynamic hand gesture classification was conducted by using a Bag-of-features (BOF) and non-linear support vector machine (SVM) methods. As the BOF does not track the order of events we use a multiscale local part model to preserve temporal context. Initial experimental results show a higher level of recognition. A voice recognition system is then used to translate voice commands complementing the hand gesture commands for the human-intuitive control of a personal service android robot for smart home or long-term healthcare environment applications.

Dynamic sign language and voice recognition for smart home interactive application

The Unified Modeling Language (UML) does not fully address the needs of some important modeling domains, including goals and non-functional requirements (NFR). However, UML can be extended and tailored through the definition of profiles. In this paper, we propose a UML profile for the Goal-oriented Requirement Language (GRL), the goal/NFR notation of the User Requirements Notation (URN), recently standardized by ITU-T as Recommendation Z.151. Our profile is based on the abstract metamodel of GRL defined in accordance with ITU-T Recommendation Z.111 (meta-metamodel). This GRL metamodel has already been successfully tested and implemented in the jUCMNav Eclipse plug-in (a URN modeling tool). The profiling approach used in this paper adheres to the guidelines for UML profile design defined in ITU-T Recommendation Z.119. The resulting profile has been implemented in a UML 2 tool, namely IBM Rational Tau 4.0, and validated with case studies. Our experience and lessons learned are also discussed.

/pdf/a-uml-profile-for-goal-oriented-modeling-i60pmgzdj4.pdf

Muhammad Rizwan Abid

Papers

Dynamic Sign Language Recognition for Smart Home Interactive Application Using Stochastic Linear Formal Grammar

Interacting with Digital Signage Using Hand Gestures

Dynamic hand gesture recognition for human-robot and inter-robot communication

Dynamic sign language and voice recognition for smart home interactive application

A UML profile for goal-oriented modeling