Showing papers on "Sketch recognition published in 2014"

Hand Gesture Recognition in Real-Time for Automotive Interfaces: A Multimodal Vision-based Approach and Evaluations

Abstract: In this paper, we develop a vision-based system that employs a combined RGB and depth descriptor to classify hand gestures. The method is studied for a human-machine interface application in the car. Two interconnected modules are employed: one that detects a hand in the region of interaction and performs user classification, and another that performs gesture recognition. The feasibility of the system is demonstrated using a challenging RGBD hand gesture data set collected under settings of common illumination variation and occlusion.

Strokelets: A Learned Multi-scale Representation for Scene Text Recognition

Abstract: Driven by the wide range of applications, scene text detection and recognition have become active research topics in computer vision. Though extensively studied, localizing and reading text in uncontrolled environments remain extremely challenging, due to various interference factors. In this paper, we propose a novel multi-scale representation for scene text recognition. This representation consists of a set of detectable primitives, termed as strokelets, which capture the essential substructures of characters at different granularities. Strokelets possess four distinctive advantages: (1) Usability: automatically learned from bounding box labels, (2) Robustness: insensitive to interference factors, (3) Generality: applicable to variant languages, and (4) Expressivity: effective at describing characters. Extensive experiments on standard benchmarks verify the advantages of strokelets and demonstrate the effectiveness of the proposed algorithm for text recognition.

Real-time exemplar-based face sketch synthesis

Abstract: This paper proposes a simple yet effective face sketch synthesis method. Similar to existing exemplar-based methods, a training dataset containing photo-sketch pairs is required, and a K-NN photo patch search is performed between a test photo and every training exemplar for sketch patch selection. Instead of using the Markov Random Field to optimize global sketch patch selection, this paper formulates face sketch synthesis as an image denoising problem which can be solved efficiently using the proposed method. Real-time performance can be obtained on a state-of-the-art GPU. Meanwhile quantitative evaluations on face sketch recognition and user study demonstrate the effectiveness of the proposed method. In addition, the proposed method can be directly extended to the temporal domain for consistent video sketch synthesis, which is of great importance in digital entertainment.

Adaptive Gesture Recognition with Variation Estimation for Interactive Systems

Abstract: This article presents a gesture recognition/adaptation system for human--computer interaction applications that goes beyond activity classification and that, as a complement to gesture labeling, characterizes the movement execution. We describe a template-based recognition method that simultaneously aligns the input gesture to the templates using a Sequential Monte Carlo inference technique. Contrary to standard template-based methods based on dynamic programming, such as Dynamic Time Warping, the algorithm has an adaptation process that tracks gesture variation in real time. The method continuously updates, during execution of the gesture, the estimated parameters and recognition results, which offers key advantages for continuous human--machine interaction. The technique is evaluated in several different ways: Recognition and early recognition are evaluated on 2D onscreen pen gestures; adaptation is assessed on synthetic data; and both early recognition and adaptation are evaluated in a user study involving 3D free-space gestures. The method is robust to noise, and successfully adapts to parameter variation. Moreover, it performs recognition as well as or better than nonadapting offline template-based methods.

RETRACTED ARTICLE: Human---computer interaction using vision-based hand gesture recognition systems: a survey

Abstract: Considerable effort has been put toward the development of intelligent and natural interfaces between users and computer systems. In line with this endeavor, several modes of information (e.g., visual, audio, and pen) that are used either individually or in combination have been proposed. The use of gestures to convey information is an important part of human communication. Hand gesture recognition is widely used in many applications, such as in computer games, machinery control (e.g., crane), and thorough mouse replacement. Computer recognition of hand gestures may provide a natural computer interface that allows people to point at or to rotate a computer-aided design model by rotating their hands. Hand gestures can be classified into two categories: static and dynamic. The use of hand gestures as a natural interface serves as a motivating force for research on gesture taxonomy, its representations, and recognition techniques. This paper summarizes the surveys carried out in human--computer interaction (HCI) studies and focuses on different application domains that use hand gestures for efficient interaction. This exploratory survey aims to provide a progress report on static and dynamic hand gesture recognition (i.e., gesture taxonomies, representations, and recognition techniques) in HCI and to identify future directions on this topic.

XKin: an open source framework for hand pose and gesture recognition using kinect

Abstract: This work targets real-time recognition of both static hand-poses and dynamic hand-gestures in a unified open-source framework. The developed solution enables natural and intuitive hand-pose recognition of American Sign Language (ASL), extending the recognition to ambiguous letters not challenged by previous work. While hand-pose recognition exploits techniques working on depth information using texture-based descriptors, gesture recognition evaluates hand trajectories in the depth stream using angular features and hidden Markov models (HMM). Although classifiers come already trained on ASL alphabet and 16 uni-stroke dynamic gestures, users are able to extend these default sets by adding their personalized poses and gestures. The accuracy and robustness of the recognition system have been evaluated using a publicly available database and across many users. The XKin open project is available online (Pedersoli, XKin libraries. https://github.com/fpeder/XKin , 2013) under FreeBSD License for researchers in human---machine interaction.

Sign language recognition: state of the art

Abstract: Sign language is used by deaf and hard hearing people to exchange information between their own community and with other people. Computer recognition of sign language deals from sign gesture acquisition and continues till text/speech generation. Sign gestures can be classified as static and dynamic. However static gesture recognition is simpler than dynamic gesture recognition but both recognition systems are important to the human community. The sign language recognition steps are described in this survey. The data acquisition, data preprocessing and transformation, feature extraction, classification and results obtained are examined. Some future directions for research in this area also suggested.

Cross-Modal Face Matching: Beyond Viewed Sketches

Abstract: Matching face images across different modalities is a challenging open problem for various reasons, notably feature heterogeneity, and particularly in the case of sketch recognition – abstraction, exaggeration and distortion. Existing studies have attempted to address this task by engineering invariant features, or learning a common subspace between the modalities. In this paper, we take a different approach and explore learning a mid-level representation within each domain that allows faces in each modality to be compared in a domain invariant way. In particular, we investigate sketch-photo face matching and go beyond the well-studied viewed sketches to tackle forensic sketches and caricatures where representations are often symbolic. We approach this by learning a facial attribute model independently in each domain that represents faces in terms of semantic properties. This representation is thus more invariant to heterogeneity, distortions and robust to mis-alignment. Our intermediate level attribute representation is then integrated synergistically with the original low-level features using CCA. Our framework shows impressive results on cross-modal matching tasks using forensic sketches, and even more challenging caricature sketches. Furthermore, we create a new dataset with \(\approx \)59, 000 attribute annotations for evaluation and to facilitate future research.

Recognizing composite sketches with digital face images via SSD dictionary

Abstract: Sketch recognition has important law enforcement applications in detecting and apprehending suspects. Compared to hand drawn sketches, software generated composite sketches are faster to create and require lesser skill sets as well as bring consistency in sketch generation. While sketch generation is one side of the problem, recognizing composite sketches with digital images is another side. This paper presents an algorithm to address the second problem, i.e. matching composite sketches with digital images. The proposed algorithm utilizes a SSD based dictionary generated via 50,000 images from the CMU Multi-PIE database. The gallery-probe feature vectors created using SSD dictionary are matched using GentleBoostKO classifier. The results on extended PRIP composite sketch database show the effectiveness of the proposed algorithm.

Human Interaction Recognition Based on the Co-occurrence of Visual Words

Abstract: This paper describes a novel methodology for automated recognition of high-level activities. A key aspect of our framework relies on the concept of co-occurring visual words for describing interactions between several persons. Motivated by the numerous success of human activity recognition methods using bag-of-words, this paradigm is extended. A 3-D XYT spatio-temporal volume is generated for each interacting person and a set of visual words is extracted to represent his activity. The interaction is then represented by the frequency of co-occurring visual words between persons. For our experiments, we used the UT-interaction dataset which contains several complex human-human interactions.

Learning Person-Specific Representations From Faces in the Wild

Abstract: Humans are natural face recognition experts, far out-performing current automated face recognition algorithms, especially in naturalistic, “in the wild” settings. However, a striking feature of human face recognition is that we are dramatically better at recognizing highly familiar faces, presumably because we can leverage large amounts of past experience with the appearance of an individual to aid future recognition. Meanwhile, the analogous situation in automated face recognition, where a large number of training examples of an individual are available, has been largely underexplored, in spite of the increasing relevance of this setting in the age of social media. Inspired by these observations, we propose to explicitly learn enhanced face representations on a per-individual basis, and we present two methods enabling this approach. By learning and operating within person-specific representations, we are able to significantly outperform the previous state-of-the-art on PubFig83, a challenging benchmark for familiar face recognition in the wild, using a novel method for learning representations in deep visual hierarchies. We suggest that such person-specific representations aid recognition by introducing an intermediate form of regularization to the problem.

Scene Text Recognition Using Structure-Guided Character Detection and Linguistic Knowledge

Abstract: Scene text recognition has inspired great interests from the computer vision community in recent years. In this paper, we propose a novel scene text-recognition method integrating structure-guided character detection and linguistic knowledge. We use part-based tree structure to model each category of characters so as to detect and recognize characters simultaneously. Since the character models make use of both the local appearance and global structure informations, the detection results are more reliable. For word recognition, we combine the detection scores and language model into the posterior probability of character sequence from the Bayesian decision view. The final word-recognition result is obtained by maximizing the character sequence posterior probability via Viterbi algorithm. Experimental results on a range of challenging public data sets (ICDAR 2003, ICDAR 2011, SVT) demonstrate that the proposed method achieves state-of-the-art performance both for character detection and word recognition.

Real-time hand gesture recognition with Kinect for playing racing video games

Abstract: This paper presents a Kinect based hand gesture recognition system that can effectively recognize both one-hand and two-hand gestures. It is robust against the disturbance of complex background and objects such as the faces and hands of other people by exploiting the depth information and carefully choosing the region of interest (ROI) in the process of tracking. The recognition module is implemented using template matching and other light weight techniques to reduce the computational complexity. In the experiments, this system is tested on real world tasks from controlling the slide show in PowerPoint to playing the highly intense racing video game Need for Speed. The practical performance confirms that our system is both effective in terms of robustness and versatility and efficient for various real-time applications.

Hand posture and gesture recognition technology

Abstract: Hand gestures that are performed by one or two hands can be categorized according to their applications into different categories including conversational, controlling, manipulative and communicative gestures. Generally, hand gesture recognition aims to identify specific human gestures and use them to convey information. The process of hand gesture recognition composes mainly of four stages: hand gesture images collection, gesture image preprocessing using some techniques including edge detection, filtering and normalization, capture the main characteristics of the gesture images and the evaluation (or classification) stage where the image is classified to its corresponding gesture class. There are many methods that have been used in the classification stage of hand gesture recognition such as Artificial Neural Networks, template matching, Hidden Markov Models and Dynamic Time Warping. This exploratory survey aims to provide a progress report on hand posture and gesture recognition technology.

Comparative study of leaf image recognition with a novel learning-based approach

Abstract: Automatic plant identification via computer vision techniques has been greatly important for a number of professionals, such as environmental protectors, land managers, and foresters. In this paper, we conduct a comparative study on leaf image recognition and propose a novel learning-based leaf image recognition technique via sparse representation (or sparse coding) for automatic plant identification. In our learning-based method, in order to model leaf images, we learn an overcomplete dictionary for sparsely representing the training images of each leaf species. Each dictionary is learned using a set of descriptors extracted from the training images in such a way that each descriptor is represented by linear combination of a small number of dictionary atoms. Moreover, we also implement a general bag-of-words (BoW) model-based recognition system for leaf images, used for comparison. We experimentally compare the two approaches and show unique characteristics of our sparse coding-based framework. As a result, efficient leaf recognition can be achieved on public leaf image dataset based on the two evaluated methods, where the proposed sparse coding-based framework can perform better.

Mechanix: A natural sketch interface tool for teaching truss analysis and free-body diagrams

Abstract: Massive open online courses, online tutoring systems, and other computer homework systems are rapidly changing engineering education by providing increased student feedback and capitalizing upon online systems' scalability. While online homework systems provide great benefits, a growing concern among engineering educators is that students are losing both the critical art of sketching and the ability to take a real system and reduce it to an accurate but simplified free-body diagram (FBD). For example, some online systems allow the drag and drop of forces onto FBDs, but they do not allow the user to sketch the FBDs, which is a vital part of the learning process. In this paper, we discuss Mechanix, a sketch recognition tool that provides an efficient means for engineering students to learn how to draw truss FBDs and solve truss problems. The system allows students to sketch FBDs into a tablet computer or by using a mouse and a standard computer monitor. Using artificial intelligence, Mechanix can determine not only the component shapes and features of the diagram but also the relationships between those shapes and features. Because Mechanix is domain specific, it can use those relationships to determine not only whether a student's work is correct but also why it is incorrect. Mechanix is then able to provide immediate, constructive feedback to students without providing final answers. Within this manuscript, we document the inner workings of Mechanix, including the artificial intelligence behind the scenes, and present studies of the effects on student learning. The evaluations have shown that Mechanix is as effective as paper-and-pencil-based homework for teaching method of joints truss analysis; focus groups with students who used the program have revealed that they believe Mechanix enhances their learning and that they are highly engaged while using it.

Using Appearance-Based Hand Features for Dynamic RGB-D Gesture Recognition

Abstract: Gesture recognition using RGB-D sensors has currently an important role in many fields such as human-computer interfaces, robotics control, and sign language recognition. However, the recognition of hand gestures under natural conditions with low spatial resolution and strong motion blur still remains an open research question. In this paper we propose an online gesture recognition method for multimodal RGB-D data. We extract multiple hand features with the assistance of body and hand masks from RGB and depth frames, and full-body features from the skeleton data. These features are classified by multiple Extreme Learning Machines on the frame level. The classifier outputs are then modeled on the sequence level and fused together to provide the final classification results for the gestures. We apply our method on the ChaLearn 2013 gesture dataset consisting of natural signs with the hand diameters in the images around 20-10 pixels. Our method achieves an 85% recognition accuracy with 20 gesture classes and can perform the recognition in real-time.

Vision based hand gesture recognition using eccentric approach for human computer interaction

Abstract: There has been growing interest in the development of new approaches and technologies for bridging the human-computer barrier. Hand gesture recognition is considered as an interaction technique having potential to communicate with machines. Human computer interaction (HCI) was never an easy task and lots of approaches are available to build such systems. Hand gesture recognition (HGR) using wearable data glove provides a solution to build a HCI system, but lags in terms of its computational time and poor interface. Pattern matching is one more solution which uses vision based techniques and provides strong interface to build HCI systems. But again, it requires complex algorithms which takes lots of computational time and hence limits its use in real time HCI applications. In this paper, we presented an eccentric approach for hand gesture recognition which is simple, fast and user independent and can be used to develop real time HCI applications. Based on proposed algorithm we built a system for Indian Sign Language recognition which converts Indian Sign numbers into text. The algorithm first captures the image of single handed gesture of speech/hearing impaired person using simple webcam and then using our proposed algorithm it classifies the gesture into its appropriate class. It uses simple logical conditions for gesture classification which make its use in real time HCI applications.

Computer Vision Based Human-Computer Interaction Using Color Detection Techniques

Abstract: A gesture-based human computer interaction allows people to control the application on windows by moving their hands through the air and make computers and devices easier to use. Existing solutions have relied on gesture recognition algorithms they needs different hard wares, often involving complicated setups limited to the research lab. Algorithms which are used so far for gesture recognition are not practical or responsive enough for real-world use, might be due to the inadequate data on which the image processing is done. As existing methods are based on gesture recognition algorithms. It needs 'ANN training' which makes whole process slow and reduces accuracy. Method we proposed is based on real time controlling the motion of mouse in windows according to the motion of hand and fingers by calculating the change in pixels values of RBG colors from a video, 'without using any ANN training' to get exact sequence of motion of hands and fingers.

Real time hand gesture recognition for computer interaction

Abstract: Hand gesture recognition is a natural and intuitive way to interact with the computer, since interactions with the computer can be increased through multidimensional use of hand gestures as compare to other input methods. The purpose of this paper is to explore three different techniques for HGR (hand gesture recognition) using finger tips detection. A new approach called “Curvature of Perimeter” is presented with its application as a virtual mouse. The system presented, uses only a webcam and algorithms which are developed using computer vision, image and the video processing toolboxes of Matlab.

Kinect-based multimodal gesture recognition using a two-pass fusion scheme

Abstract: We present a new framework for multimodal gesture recognition that is based on a two-pass fusion scheme. In this, we deal with a demanding Kinect-based multimodal dataset, which was introduced in a recent gesture recognition challenge. We employ multiple modalities, i.e., visual cues, such as colour and depth images, as well as audio, and we specifically extract feature descriptors of the hands' movement, handshape, and audio spectral properties. Based on these features, we statistically train separate unimodal gesture-word models, namely hidden Markov models, explicitly accounting for the dynamics of each modality. Multimodal recognition of unknown gesture sequences is achieved by combining these models in a late, two-pass fusion scheme that exploits a set of unimodally generated n-best recognition hypotheses. The proposed scheme achieves 88.2% gesture recognition accuracy in the Kinect-based multimodal dataset, outperforming all recently published approaches on the same challenging multimodal gesture recognition task.

Local context-based recognition of sketched diagrams

Abstract: We present a new methodology aimed at the design and implementation of a framework for sketch recognition enabling the recognition and interpretation of diagrams. The diagrams may contain different types of sketched graphic elements such as symbols, connectors, and text. Once symbols are distinguished from connectors and identified, the recognition proceeds by identifying the local context of each symbol. This is seen as the symbol interface exposed to the rest of the diagram and includes predefined attachment areas on each symbol. The definition of simple constraints on the local context of each symbol allows to greatly simplify the definition of the visual grammar, which is used only for further refinement and interpretation of the set of acceptable diagrams. We demonstrate the potential of the methodology using flowcharts and binary trees as examples. HighlightsWe present a new methodology aimed at the design and implementation of a framework for sketch recognition.The recognition proceeds by identifying the local context of each symbol in a diagram.Our methodology allows greatly simplifying the definition of the visual grammar.We demonstrate the potential of the methodology using flowcharts as an example and binary trees.

Generic system for human-computer gesture interaction

Abstract: Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of vision-based interaction systems can be the same for all applications and thus facilitate the implementation. In order to test the proposed solutions, three prototypes were implemented. For hand posture recognition, a SVM model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications.

Accelerometer-based hand gesture recognition system for interaction in digital TV

Abstract: This paper presents the design and implementation of a system of accelerometer-based hand gesture recognition This system will be embedded within a modern remote control to improve human-machine interaction in the context of digital TV of Argentina As the recognition of hand gestures is a pattern classification problem, two techniques based on artificial neural networks are explored: multilayer perceptron and support vector machine This is performed in order to compare results and select the tool that best fits the problem Jointly, signal digital processing techniques are used for preprocessing and adapting of the input signals to pattern recognition models A gestural vocabulary of 8 types of gestures was used, which was also used by other similar works in order to compare results An appropriate trade-off between the classifier recognition precision and resource utilization of the hardware platform is required in order to implement the solution within an embedded system The obtained results of precision and utilization of resources are excellent

Face Sketch Recognition from Local Features

Abstract: Systems for face sketch recognition are very important for law enforcement agencies. These systems can help to locate or narrow down potential suspects. Recently, various methods were proposed to address this problem, but there is no clear comparison of their performance. In this paper is proposed a new approach for photo/sketch recognition based on the Local Feature-based Discriminant Analysis (LFDA) method. This new approach was tested and compared with its predecessors using three differents datasets and also adding an extra gallery of 10,000 photos to extend the gallery. Experiments using the CUFS and CUFSF databases show that our approach outperforms the state-of-the-art approaches. Our approach also shows good results with forensic sketches. The limitation with this dataset is its very small size. By increasing the training dataset, the accuracy of our approach increases, as it was demonstrated by our experiments.

Hidden Markov models based dynamic hand gesture recognition with incremental learning method

Abstract: This paper proposes a real-time dynamic hand gesture recognition system based on Hidden Markov Models with incremental learning method (IL-HMMs) to provide natural human-computer interaction. The system is divided into four parts: hand detecting and tracking, feature extraction and vector quantization, HMMs training and hand gesture recognition, incremental learning. After quantized hand gesture vector being recognized by HMMs, incremental learning method is adopted to modify the parameters of corresponding recognized model to make itself more adaptable to the coming new gestures. Experiment results show that comparing with traditional one, the proposed system can obtain better recognition rates.