Showing papers on "Sketch recognition published in 2013"

Robust Part-Based Hand Gesture Recognition Using Kinect Sensor

Abstract: The recently developed depth sensors, e.g., the Kinect sensor, have provided new opportunities for human-computer interaction (HCI). Although great progress has been made by leveraging the Kinect sensor, e.g., in human body tracking, face recognition and human action recognition, robust hand gesture recognition remains an open problem. Compared to the entire human body, the hand is a smaller object with more complex articulations and more easily affected by segmentation errors. It is thus a very challenging problem to recognize hand gestures. This paper focuses on building a robust part-based hand gesture recognition system using Kinect sensor. To handle the noisy hand shapes obtained from the Kinect sensor, we propose a novel distance metric, Finger-Earth Mover's Distance (FEMD), to measure the dissimilarity between hand shapes. As it only matches the finger parts while not the whole hand, it can better distinguish the hand gestures of slight differences. The extensive experiments demonstrate that our hand gesture recognition system is accurate (a 93.2% mean accuracy on a challenging 10-gesture dataset), efficient (average 0.0750 s per frame), robust to hand articulations, distortions and orientation or scale changes, and can work in uncontrolled environments (cluttered backgrounds and lighting conditions). The superiority of our system is further demonstrated in two real-life HCI applications.

Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models

Abstract: This work presents Sketch2Scene, a framework that automatically turns a freehand sketch drawing inferring multiple scene objects to semantically valid, well arranged scenes of 3D models. Unlike the existing works on sketch-based search and composition of 3D models, which typically process individual sketched objects one by one, our technique performs co-retrieval and co-placement of 3D relevant models by jointly processing the sketched objects. This is enabled by summarizing functional and spatial relationships among models in a large collection of 3D scenes as structural groups. Our technique greatly reduces the amount of user intervention needed for sketch-based modeling of 3D scenes and fits well into the traditional production pipeline involving concept design followed by 3D modeling. A pilot study indicates that it is promising to use our technique as an alternative but more efficient tool of standard 3D modeling for 3D scene construction.

Coupled Dictionary and Feature Space Learning with Applications to Cross-Domain Image Synthesis and Recognition

Abstract: Cross-domain image synthesis and recognition are typically considered as two distinct tasks in the areas of computer vision and pattern recognition. Therefore, it is not clear whether approaches addressing one task can be easily generalized or extended for solving the other. In this paper, we propose a unified model for coupled dictionary and feature space learning. The proposed learning model not only observes a common feature space for associating cross-domain image data for recognition purposes, the derived feature space is able to jointly update the dictionaries in each image domain for improved representation. This is why our method can be applied to both cross-domain image synthesis and recognition problems. Experiments on a variety of synthesis and recognition tasks such as single image super-resolution, cross-view action recognition, and sketch-to-photo face recognition would verify the effectiveness of our proposed learning model.

Matching Composite Sketches to Face Photos: A Component-Based Approach

Abstract: The problem of automatically matching composite sketches to facial photographs is addressed in this paper. Previous research on sketch recognition focused on matching sketches drawn by professional artists who either looked directly at the subjects (viewed sketches) or used a verbal description of the subject's appearance as provided by an eyewitness (forensic sketches). Unlike sketches hand drawn by artists, composite sketches are synthesized using one of the several facial composite software systems available to law enforcement agencies. We propose a component-based representation (CBR) approach to measure the similarity between a composite sketch and mugshot photograph. Specifically, we first automatically detect facial landmarks in composite sketches and face photos using an active shape model (ASM). Features are then extracted for each facial component using multiscale local binary patterns (MLBPs), and per component similarity is calculated. Finally, the similarity scores obtained from individual facial components are fused together, yielding a similarity score between a composite sketch and a face photo. Matching performance is further improved by filtering the large gallery of mugshot images using gender information. Experimental results on matching 123 composite sketches against two galleries with 10,123 and 1,316 mugshots show that the proposed method achieves promising performance (rank-100 accuracies of 77.2% and 89.4%, respectively) compared to a leading commercial face recognition system (rank-100 accuracies of 22.8% and 52.0%) and densely sampled MLBP on holistic faces (rank-100 accuracies of 27.6% and 10.6%). We believe our prototype system will be of great value to law enforcement agencies in apprehending suspects in a timely fashion.

Scene Text Recognition Using Part-Based Tree-Structured Character Detection

Abstract: Scene text recognition has inspired great interests from the computer vision community in recent years. In this paper, we propose a novel scene text recognition method using part-based tree-structured character detection. Different from conventional multi-scale sliding window character detection strategy, which does not make use of the character-specific structure information, we use part-based tree-structure to model each type of character so as to detect and recognize the characters at the same time. While for word recognition, we build a Conditional Random Field model on the potential character locations to incorporate the detection scores, spatial constraints and linguistic knowledge into one framework. The final word recognition result is obtained by minimizing the cost function defined on the random field. Experimental results on a range of challenging public datasets (ICDAR 2003, ICDAR 2011, SVT) demonstrate that the proposed method outperforms state-of-the-art methods significantly both for character detection and word recognition.

Real-Time Sign Language Recognition Using a Consumer Depth Camera

Abstract: Gesture recognition remains a very challenging task in the field of computer vision and human computer interaction (HCI). A decade ago the task seemed to be almost unsolvable with the data provided by a single RGB camera. Due to recent advances in sensing technologies, such as time-of-flight and structured light cameras, there are new data sources available, which make hand gesture recognition more feasible. In this work, we propose a highly precise method to recognize static gestures from a depth data, provided from one of the above mentioned devices. The depth images are used to derive rotation-, translation- and scale-invariant features. A multi-layered random forest (MLRF) is then trained to classify the feature vectors, which yields to the recognition of the hand signs. The training time and memory required by MLRF are much smaller, compared to a simple random forest with equivalent precision. This allows to repeat the training procedure of MLRF without significant effort. To show the advantages of our technique, we evaluate our algorithm on synthetic data, on publicly available dataset, containing 24 signs from American Sign Language(ASL) and on a new dataset, collected using recently appeared Intel Creative Gesture Camera.

Geosemantic Snapping for Sketch-Based Modeling

Abstract: Modeling 3D objects from sketches is a process that requires several challenging problems including segmentation, recognition and reconstruction. Some of these tasks are harder for humans and some are harder for the machine. At the core of the problem lies the need for semantic understanding of the shape’s geometry from the sketch. In this paper we propose a method to model 3D objects from sketches by utilizing humans specifically for semantic tasks that are very simple for humans and extremely difficult for the machine, while utilizing the machine for tasks that are harder for humans. The user assists recognition and segmentation by choosing and placing specific geometric primitives on the relevant parts of the sketch. The machine first snaps the primitive to the sketch by fitting its projection to the sketch lines, and then improves the model globally by inferring geosemantic constraints that link the different parts. The fitting occurs in real-time, allowing the user to be only as precise as needed to have a good starting configuration for this non-convex optimization problem. We evaluate the accessibility of our approach with a user study.

Sketch recognition by ensemble matching of structured features

Abstract: Sketch recognition aims to automatically classify human hand sketches of objects into known categories. This has become increasingly a desirable capability due to recent advances in human computer interaction on portable devices. The problem is nontrivial because of the sparse and abstract nature of hand drawings as compared to photographic images of objects, compounded by a highly variable degree of details in human sketches. To this end, we present a method for the representation and matching of sketches by exploiting not only local features but also global structures of sketches, through a star graph based ensemble matching strategy. Different local feature representations were evaluated using the star graph model to demonstrate the effectiveness of the ensemble matching of structured features. We further show that by encapsulating holistic structure matching and learned bag-of-features models into a single framework, notable recognition performance improvement over the state-of-the-art can be observed. Extensive comparative experiments were carried out using the currently largest sketch dataset released by Eitz et al. [15], with over 20,000 sketches of 250 object categories generated by AMT (Amazon Mechanical Turk) crowd-sourcing.

A Survey on Hand Gesture Recognition

Abstract: Hand gesture recognition has become one of the key techniques of human-computer interaction (HCI) Many researchers are devoted in this field In this paper, firstly the history of hand gesture recognition is discussed and the technical difficulties are also enumerated Then, we analyze the definition of hand gesture and introduce the basic principle of it The approaches for hand gesture recognition, such as vision-based, glove-based and depth-based, are contrasted briefly in this paper But the former two methods are too simple and not natural enough Currently, the new finger identification and hand gesture recognition technique with Kinect depth data is the most popular research direction Finally, we discuss the application prospective of hand gesture recognition based on Kinect

Most Probable Longest Common Subsequence for Recognition of Gesture Character Input

Abstract: This paper presents a technique for trajectory classification with applications to dynamic free-air hand gesture recognition. Such gestures are unencumbered and drawn in free air. Our approach is an extension to the longest common subsequence (LCS) classification algorithm. A learning preprocessing stage is performed to create a probabilistic 2-D template for each gesture, which allows taking into account different trajectory distortions with different probabilities. The modified LCS, termed the most probable LCS (MPLCS), is developed to measure the similarity between the probabilistic template and the hand gesture sample. The final decision is based on the length and probability of the extracted subsequence. Validation tests using a cohort of gesture digits from video-based capture show that the approach is promising with a recognition rate of more than 98 % for video stream preisolated digits. The MPLCS algorithm can be integrated into a gesture recognition interface to facilitate gesture character input. This can greatly enhance the usability of such interfaces.

Static hand gesture recognition based on HOG characters and support vector machines

Abstract: Gesture recognition technology has important significance in the field of human-computer interaction (HCI), the gesture recognition technology which is based on visual is sensitive to the impact of the experimental environment lighting, and so, the recognition result will produce a greater change; it makes this technology one of the most challenging topics. HOG feature which is successfully applied to pedestrian detection is operating on the local grid unit of image, so it can maintain a good invariance on geometric and optical deformation. In this paper, we extracted the gradient direction histogram (HOG) features of gestures, then, a Support Vector Machines is used to train these feature vectors, at testing time, a decision is taken using the previously learned SVMs, and compared the same gesture recognition rate in different light conditions. Experimental results show that the HOG feature extraction and multivariate SVM classification methods has a high recognition rate, and the system has a better robustness for the illumination.

Real-time and continuous hand gesture spotting: An approach based on artificial neural networks

Abstract: New and more natural human-robot interfaces are of crucial interest to the evolution of robotics. This paper addresses continuous and real-time hand gesture spotting, i.e., gesture segmentation plus gesture recognition. Gesture patterns are recognized by using artificial neural networks (ANNs) specifically adapted to the process of controlling an industrial robot. Since in continuous gesture recognition the communicative gestures appear intermittently with the non-communicative, we are proposing a new architecture with two ANNs in series to recognize both kinds of gesture. A data glove is used as interface technology. Experimental results demonstrated that the proposed solution presents high recognition rates (over 99% for a library of ten gestures and over 96% for a library of thirty gestures), low training and learning time and a good capacity to generalize from particular situations.

Online RGB-D gesture recognition with extreme learning machines

Abstract: Gesture recognition is needed in many applications such as human-computer interaction and sign language recognition. The challenges of building an actual recognition system do not lie only in reaching an acceptable recognition accuracy but also with requirements for fast online processing. In this paper, we propose a method for online gesture recognition using RGB-D data from a Kinect sensor. Frame-level features are extracted from RGB frames and the skeletal model obtained from the depth data, and then classified by multiple extreme learning machines. The outputs from the classifiers are aggregated to provide the final classification results for the gestures. We test our method on the ChaLearn multi-modal gesture challenge data. The results of the experiments demonstrate that the method can perform effective multi-class gesture recognition in real-time.

Towards machine learning based design pattern recognition

Abstract: Software design patterns are abstract descriptions of best practice solutions for recurring design problems. The information about which design pattern is implemented where in a software design is very helpful and important for software maintenance and evolution. This information is usually lost due to poor, obsolete or lack of documentation, which raises the importance of automatic recognition techniques. However, their vague and abstract nature allows them to be implemented in various ways, which gives them resistance to be automatically and accurately recognized. This paper presents the first recognition approach to be solely based on machine learning methods. We build a training dataset by using several existing recognition tools and we use feature selection methods to select the input feature vectors. Artificial neural networks are then trained to perform the whole recognition process. Our approach is evaluated by conducting an experiment to recognize six design patterns in an open source application.

Face expression recognition: A brief overview of the last decade

Abstract: The huge research effort in the field of face expression recognition (FER) technology is justified by the potential applications in multiple domains: computer science, engineering, psychology, neuroscience, to name just a few. Obviously, this generates an impressive number of scientific publications. The aim of this paper is to identify key representative approaches for facial expression recognition research in the past ten years (2003-2012).

Hidden Markov model for human to computer interaction: a study on human hand gesture recognition

Abstract: Human hand recognition plays an important role in a wide range of applications ranging from sign language translators, gesture recognition, augmented reality, surveillance and medical image processing to various Human Computer Interaction (HCI) domains. Human hand is a complex articulated object consisting of many connected parts and joints. Therefore, for applications that involve HCI one can find many challenges to establish a system with high detection and recognition accuracy for hand posture and/or gesture. Hand posture is defined as a static hand configuration without any movement involved. Meanwhile, hand gesture is a sequence of hand postures connected by continuous motions. During the past decades, many approaches have been presented for hand posture and/or gesture recognition. In this paper, we provide a survey on approaches which are based on Hidden Markov Models (HMM) for hand posture and gesture recognition for HCI applications.

Modeling Flowchart Structure Recognition as a Max-Sum Problem

Abstract: This work deals with the on-line recognition of hand-drawn graphical sketches with structure. We present a novel approach, in which the search for a suitable interpretation of the input is formulated as a combinatorial optimization task - the max-sum problem. The recognition pipeline consists of two main stages. First, groups of strokes possibly representing symbols of a sketch (symbol candidates) are segmented and relations between them are detected. Second, a combination of symbol candidates best fitting the input is chosen by solving the optimization problem. We focused on flowchart recognition. Training and testing of our method was done on a freely available benchmark database. We correctly segmented and recognized 82.7% of the symbols having 31.5% of the diagrams recognized without any error. It indicates that our approach has promising potential and can compete with the state-of-the-art methods.

Hand Gesture Recognition using Computer Vision

Abstract: The use of the gesture system in our daily life as a natural human-human interaction has inspired the researchers to simulate and utilize this gift in human-machine interaction which is appealing and can take place the bore interaction ones that existed such as television, radio, and various home appliances as well as virtual reality will worth and deserve its name. This kind of interaction ensures promising and satisfying outcomes if applied in systematic approach, and supports unadorned human hand when transferring the message to these devices which is easiest, comfort and desired rather than the communication that requires frills to deliver the message to such devices. With the rapid emergence of 3d applications and virtual environments in computer system the need for a new type of interaction device arises. This is because the traditional devices such as mouse, keyboard and joystick become inefficient and cumbersome within this virtual environments.in other words evolution of user interfaces shapes the change in the HumanComputer Interaction (HCI).Intuitive and naturalness characteristics of “Hand Gestures” in the HCI have been the driving force and motivation to develop an interaction device which can replace current unwieldy tools. A survey on the methods of analysing, modelling and recognizing hand gestures in the context of the HCI is provided in this paper.

Hand gesture recognition: An overview

Abstract: Hand gesture recognition has been applied to many fields in recent years, especially in man-machine interaction (MMI) area, which is regarded as a more natural and flexible input. In this paper, an overview of hand gesture recognition research up to date is presented, which includes common stages of hand gesture recognition, common methods and technique of each stage, the state of the recent research and summaries of some successful hand gesture recognition models.

Position-invariant, real-time gesture recognition based on dynamic time warping

Abstract: To achieve an improved human-robot interaction it is necessary to allow the human participant to interact with the robot in a natural way. In this work, a gesture recognition algorithm, based on dynamic time warping, was implemented with a use-case scenario of natural interaction with a mobile robot. Inputs are gesture trajectories obtained using a Microsoft Kinect sensor. Trajectories are stored in the person's frame of reference. Furthermore, the recognition is position-invariant, meaning that only one learned sample is needed to recognize the same gesture performed at another position in the gestural space. In experiments, a set of gestures for a robot waiter was used to train the gesture recognition algorithm. The experimental results show that the proposed modifications of the standard gesture recognition algorithm improve the robustness of the recognition.

Robot anticipation of human intentions through continuous gesture recognition

Abstract: In this paper, we propose a method to recognize human body movements and we combine it with the contextual knowledge of human-robot collaboration scenarios provided by an object affordances framework that associates actions with its effects and the objects involved in them. The aim is to equip humanoid robots with action prediction capabilities, allowing them to anticipate effects as soon as a human partner starts performing a physical action, thus enabling interactions between man and robot to be fast and natural. We consider simple actions that characterize a human-robot collaboration scenario with objects being manipulated on a table: inspired from automatic speech recognition techniques, we train a statistical gesture model in order to recognize those physical gestures in real time. Analogies and differences between the two domains are discussed, highlighting the requirements of an automatic gesture recognizer for robots in order to perform robustly and in real time.

Geometric Feature Based Face-Sketch Recognition

Abstract: This paper presents a novel facial sketch image or face-sketch recognition approach based on facial feature extraction. To recognize a face-sketch, we have concentrated on a set of geometric face features like eyes, nose, eyebrows, lips, etc and their length and width ratio because it is difficult to match photos and sketches because they belong to two different modalities. In this system, first the facial features/components from training images are extracted, then ratios of length, width, and area etc. are calculated and those are stored as feature vectors for individual images. After that the mean feature vectors are computed and subtracted from each feature vector for centering of the feature vectors. In the next phase, feature vector for the incoming probe face-sketch is also computed in similar fashion. Here, K-NN classifier is used to recognize probe face-sketch. It is experimentally verified that the proposed method is robust against faces are in a frontal pose, with normal lighting and neutral expression and have no occlusions. The experiment has been conducted with 80 male and female face images from different face databases. It has useful applications for both law enforcement and digital entertainment.

Human computer interaction using face and gesture recognition

Abstract: In this paper, we present a face and gesture recognition based human-computer interaction (HCI) system using a single video camera. Different from the conventional communication methods between users and machines, we combine head pose and hand gesture to control the equipment. We can identify the position of the eyes and mouth, and use the facial center to estimate the pose of the head. Two new methods are presented in this paper: automatic gesture area segmentation and orientation normalization of the hand gesture. It is not mandatory for the user to keep gestures in upright position, the system segments and normalizes the gestures automatically. The experiment shows this method is very accurate with gesture recognition rate of 93.6%. The user can control multiple devices, including robots simultaneously through a wireless network.

Sketching by perceptual grouping

Abstract: Sketch is used for rendering the visual world since prehistoric times, and has become ubiquitous nowadays with the increasing availability of touchscreens on portable devices. However, how to automatically map images to sketches, a problem that has profound implications on applications such as sketch-based image retrieval, still remains open. In this paper, we propose a novel method that draws a sketch automatically from a single natural image. Sketch extraction is posed within an unified contour grouping framework, where perceptual grouping is first used to form contour segment groups, followed by a group-based contour simplification method that generate the final sketches. In our experiment, for the first time we pose sketch evaluation as a sketch-based object recognition problem and the results validate the effectiveness of our system over the state-of-the-arts alternatives.

KimCHI: a sketch-based developmental skill classifier to enhance pen-driven educational interfaces for children

Abstract: Sketching is one of the many valuable lifelong skills that children require in their overall development, and many educational psychologists manually analyze children's sketches to assess their developmental progress. The disadvantages of manual assessment are that it is time-consuming and prone to human error and bias, so intelligent sketching interfaces have strong potential in automating this process. Unfortunately, current sketch recognition techniques concentrate solely on recognizing the meaning of sketches, rather than the sketcher's developmental skill; and do not perform well on children's sketched input, as most are trained on and developed for adult's sketches. We introduce our proposed solution called KimCHI, a specialized sketch classification technique which utilizes a sketching interface for assessing the developmental skills of children from their sketches. Our approach relies on sketch feature selection to automatically classify the developmental progress of children's sketches as either developmental or mature. We evaluated our classifiers through a user study, and our classifiers were able to differentiate the users' development skill and gender with reasonable accuracy. We subsequently created an initial sketching interface utilizing our specialized classifier called EasySketch for demonstrating educational applications to assist children in developing their sketching skills.

Proceedings of the 8th International Conference on Computer Recognition Systems CORES 2013

Abstract: The computer recognition systems are nowadays one of the most promising directions in artificial intelligence. This book is the most comprehensive study of this field. It contains a collection of 86 carefully selected articles contributed by experts of pattern recognition. It reports on current research with respect to both methodology and applications. In particular, it includes the following sections:Biometrics Data Stream Classification and Big Data AnalyticsFeatures, learning, and classifiers Image processing and computer vision Medical applications Miscellaneous applications Pattern recognition and image processing in roboticsSpeech and word recognitionThis book is a great reference tool for scientists who deal with the problems of designing computer pattern recognition systems. Its target readers can be the as well researchers as students of computer science, artificial intelligence or robotics.

Gesture Recognition by Using Kinect Skeleton Tracking System

Abstract: With the rapid development of computer technology, these years has witnessed the rapid improvement of the approaches of human computer interaction. How to make the process more natural to the users has become a hot research topic. Nowadays, more and more researchers pay attention to gesture recognition, especially when kinect is invented by Microsoft. Because it is an ordinary and natural way in which we interact with each other. This paper put forward a novel method in which we use an innovative way to abstract and notate the features of two-hand gestures and build Coupled Hidden Markov Models for two-hand gesture recognition. This method can recognize two-hand gestures. Our experiment results demonstrate that it is an effective approach for two-hand gestures recognition.

Hand gesture recognition using fourier descriptors

Abstract: Accurate, real-time hand gesture recognition is a challenging and crucial task due to the need of more natural human-computer interaction methods. The major problem lies in fining a good compromise between the accuracy of recognition and the computational load for the algorithm to run in real-time. In this paper we propose a method for static hand gesture recognition using Fourier descriptors for feature extraction with different classifiers. Fourier descriptors have the advantage of giving a set of features that are invariant to rotation, translation and scaling. They are also efficient in terms of speed as they only use a small number of points from the entire image. The proposed method is evaluated using images from the Cambridge Hand Gesture Dataset at different number of features and different classifiers. The effectiveness of the method is shown through simulation results.

Human Action Recognition Based on Depth Images from Microsoft Kinect

Abstract: Human action recognition is very important in human computer interaction. In this article, we present a new method of recognizing human actions by using Microsoft Kinect sensor, k-means clustering and Hidden Markov Models (HMMs). Kinect is able to generate human skeleton information from depth images, in addition, features representing specific body parts are generated from the skeleton information and are used for recording actions. Then k-means clustering assigns the features into clusters and HMMs analyze the relationship between these clusters. By doing this, we achieved action learning and recognition. According to our experimental results, the average accuracy was 91.4 %.