scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Leap signature recognition using HOOF and HOT features

TL;DR: This research proposes a new biometric modality using a Leap Motion device that combines an adaptation of 3D Histogram of Oriented Optical Flow and a new feature descriptor, termed as Histogramof Oriented Trajectories.
Abstract: With the growing need for secure authentication, there is an increasing interest in establishing newer biometric modalities that are verifiable in a fast manner with as few associated complexities as possible. In this research, we propose a new biometric modality using a Leap Motion device. The Leap signature is created by an individual in three-dimensional space in absence of any feedback from objects or surfaces. The proposed framework combines an adaptation of 3D Histogram of Oriented Optical Flow and a new feature descriptor, termed as Histogram of Oriented Trajectories. Experiments are performed on the IIITD Leap Signature Database, which consists of 900 samples from 60 subjects. The results are combined with a four-patch local binary pattern based face verification algorithm. An accuracy of over 91% is achieved on this database, with rate of successful spoofing attempts being approximately 1.4%.
Citations
More filters
Journal ArticleDOI
TL;DR: This paper shows how to jointly exploit the two types of sensors for accurate gesture recognition by introducing a set of novel feature descriptors both for the Leap Motion and for depth data.
Abstract: Novel 3D acquisition devices like depth cameras and the Leap Motion have recently reached the market. Depth cameras allow to obtain a complete 3D description of the framed scene while the Leap Motion sensor is a device explicitly targeted for hand gesture recognition and provides only a limited set of relevant points. This paper shows how to jointly exploit the two types of sensors for accurate gesture recognition. An ad-hoc solution for the joint calibration of the two devices is firstly presented. Then a set of novel feature descriptors is introduced both for the Leap Motion and for depth data. Various schemes based on the distances of the hand samples from the centroid, on the curvature of the hand contour and on the convex hull of the hand shape are employed and the use of Leap Motion data to aid feature extraction is also considered. The proposed feature sets are fed to two different classifiers, one based on multi-class SVMs and one exploiting Random Forests. Different feature selection algorithms have also been tested in order to reduce the complexity of the approach. Experimental results show that a very high accuracy can be obtained from the proposed method. The current implementation is also able to run in real-time.

197 citations


Cites methods from "Leap signature recognition using HO..."

  • ...The sensor has also been used for signature recognition using features based on the optical flow and on the trajectories in a recent work [19]....

    [...]

Journal ArticleDOI
TL;DR: It has been observed that, accuracies can be improved if data from both sensors are fused as compared to single sensor-based recognition, and results are combined to boost-up the recognition performance.

133 citations

Journal ArticleDOI
TL;DR: This paper has performed 3-D text recognition using hidden Markov model (HMM) and bidirectional long short-term memory neural networks (BLSTM-NNs) and created a data set consisting of 560 Latin sentences drawn by ten participants using Leap motion sensor for experiments.
Abstract: Recognition of 3-D texts drawn by fingers using Leap motion sensor can be challenging for existing text recognition frameworks. The texts sensed by Leap motion device are different from traditional offline and on-line writing systems. This is because of frequent jitters and non-uniform character sizes while writing using Leap motion interface. Moreover, because of air writing, characters, words, and lines are usually connected by continuous stroke that makes it difficult to recognize. In this paper, we present a study of segmentation and recognition of text recorded using Leap motion sensor. The segmentation task of continuous text into words is performed using a heuristic analysis of stroke length between two successive words. Next, the recognition of each segmented word is performed using sequential classifiers. In this paper, we have performed 3-D text recognition using hidden Markov model (HMM) and bidirectional long short-term memory neural networks (BLSTM-NNs). We have created a data set consisting of 560 Latin sentences drawn by ten participants using Leap motion sensor for experiments. An accuracy of 78.2% has been obtained in word segmentation, whereas 86.88% and 81.25% accuracies have been recorded in word recognition using BLSTM-NN and HMM classifiers, respectively.

57 citations


Cites methods from "Leap signature recognition using HO..."

  • ...The motion sensor is being successfully used by various researchers to develop applications such as 3D games [8], [31], security [26], upper limb rehabilitation [7], palm rehabilitation [34], humancomputer-interface [28], word segmentation [1], handwriting recognition [36], etc....

    [...]

Book ChapterDOI
24 Oct 2016
TL;DR: A hand gesture recognition system using near-infrared imagery acquired by a Leap Motion sensor is proposed, directly characterizes the hand gesture by computing a global image descriptor, called Depth Spatiograms of Quantized Patterns, without any hand segmentation stage.
Abstract: Hand gestures are one of the main alternatives for Human-Computer Interaction. For this reason, a hand gesture recognition system using near-infrared imagery acquired by a Leap Motion sensor is proposed. The recognition system directly characterizes the hand gesture by computing a global image descriptor, called Depth Spatiograms of Quantized Patterns, without any hand segmentation stage. To deal with the high dimensionality of the image descriptor, a Compressive Sensing framework is applied, obtaining a manageable image feature vector that almost preserves the original information. Finally, the resulting reduced image descriptors are analyzed by a set of Support Vectors Machines to identify the performed gesture independently of the precise hand location in the image. Promising results have been achieved using a new hand-based near-infrared database.

49 citations


Cites background from "Leap signature recognition using HO..."

  • ...In [17], the signature of a person is proposed to check his identity by processing skeleton-derived trajectories....

    [...]

Proceedings ArticleDOI
01 Dec 2017
TL;DR: Fast and highly accurate gesture recognition systems based on long short-term memory (LSTM) and convolutional neural networks that are trained to process input sequences of 3D hand positions and velocities acquired from infrared sensors are presented.
Abstract: Hand gestures provide a natural, non-verbal form of communication that can augment or replace other communication modalities such as speech or writing. Along with voice commands, hand gestures are becoming the primary means of interaction in games, augmented reality, and virtual reality platforms. Recognition accuracy, flexibility, and computational cost are some of the primary factors that can impact the incorporation of hand gestures in these new technologies, as well as their subsequent retrieval from multimodal corpora. In this paper, we present fast and highly accurate gesture recognition systems based on long short-term memory (LSTM) and convolutional neural networks (CNN) that are trained to process input sequences of 3D hand positions and velocities acquired from infrared sensors. When evaluated on real time recognition of six types of hand gestures, the proposed architectures obtain 97% F-measure, demonstrating a significant potential for practical applications in novel human-computer interfaces.

40 citations


Cites methods from "Leap signature recognition using HO..."

  • ...[6] propose a biometric authentication method that uses SVMs to classify the hand-skeletal information associated with a 3D hand signature pattern....

    [...]

References
More filters
Book
Vladimir Vapnik1
01 Jan 1995
TL;DR: Setting of the learning problem consistency of learning processes bounds on the rate of convergence ofLearning processes controlling the generalization ability of learning process constructing learning algorithms what is important in learning theory?
Abstract: Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.

40,147 citations

Book
01 Jan 1973

20,541 citations

01 Jan 1995

4,292 citations


"Leap signature recognition using HO..." refers methods in this paper

  • ...The HOOF features are used as input to SVM classifier [8] with Radial Basis Function kernel for matching and the HOT features are matched using Naive Bayes classifier [9]....

    [...]

  • ...The selection of this classifier is validated by comparing with the feature level fusion scores for Support Vector Machine with RBF kernel [8], Neural Network with sigmoid activation function [9], and Naive Bayes classifier [9]....

    [...]

Proceedings ArticleDOI
20 Jun 2009
TL;DR: This paper proposes to represent each frame of a video using a histogram of oriented optical flow (HOOF) and to recognize human actions by classifying HOOF time-series, and proposes a generalization of the Binet-Cauchy kernels to nonlinear dynamical systems (NLDS) whose output lives in a non-Euclidean space.
Abstract: System theoretic approaches to action recognition model the dynamics of a scene with linear dynamical systems (LDSs) and perform classification using metrics on the space of LDSs, e.g. Binet-Cauchy kernels. However, such approaches are only applicable to time series data living in a Euclidean space, e.g. joint trajectories extracted from motion capture data or feature point trajectories extracted from video. Much of the success of recent object recognition techniques relies on the use of more complex feature descriptors, such as SIFT descriptors or HOG descriptors, which are essentially histograms. Since histograms live in a non-Euclidean space, we can no longer model their temporal evolution with LDSs, nor can we classify them using a metric for LDSs. In this paper, we propose to represent each frame of a video using a histogram of oriented optical flow (HOOF) and to recognize human actions by classifying HOOF time-series. For this purpose, we propose a generalization of the Binet-Cauchy kernels to nonlinear dynamical systems (NLDS) whose output lives in a non-Euclidean space, e.g. the space of histograms. This can be achieved by using kernels defined on the original non-Euclidean space, leading to a well-defined metric for NLDSs. We use these kernels for the classification of actions in video sequences using (HOOF) as the output of the NLDS. We evaluate our approach to recognition of human actions in several scenarios and achieve encouraging results.

610 citations


"Leap signature recognition using HO..." refers background in this paper

  • ...Index Terms— 3D signature, Leap, HOOF, Histogram of Oriented Trajectories...

    [...]

01 Oct 2008
TL;DR: This paper explores how well this performance carries over to the related task of multi-option face identification, specifically on the Labeled Faces in the Wild (LFW) image set, and seeks to compare the performance of similarity learning methods to descriptor based methods.
Abstract: Recent methods for learning similarity between images have presented impressive results in the problem of pair matching (same/notsame classification) of face images. In this paper we explore how well this performance carries over to the related task of multi-option face identification, specifically on the Labeled Faces in the Wild (LFW) image set. In addition, we seek to compare the performance of similarity learning methods to descriptor based methods. We present the following results: (1) Descriptor-Based approaches that efficiently encode the appearance of each face image as a vector outperform the leading similarity based method in the task of multi-option face identification. (2) Straightforward use of Euclidean distance on the descriptor vectors performs somewhat worse than the similarity learning methods on the task of pair matching. (3) Adding a learning stage, the performance of descriptor based methods matches and exceeds that of similarity methods on the pair matching task. (4) A novel patch based descriptor we propose is able to improve the performance of the successful Local Binary Pattern (LBP) descriptor in both multi-option identification and same/not-same classification.

504 citations


"Leap signature recognition using HO..." refers methods in this paper

  • ...The Four-Patch Local Binary Patterns (FPLBP) algorithm [10] is used for face verification and Fig....

    [...]