Showing papers on "Sketch recognition published in 2008"

PaleoSketch: accurate primitive sketch recognition and beautification

Abstract: Sketching is a natural form of human communication and has become an increasingly popular tool for interacting with user interfaces. In order to facilitate the integration of sketching into traditional user interfaces, we must first develop accurate ways of recognizing users' intentions while providing feedback to catch recognition problems early in the sketching process. One approach to sketch recognition has been to recognize low-level primitives and then hierarchically construct higher-level shapes based on geometric constraints defined by the user, however, current low-level recognizers only handle a few number of primitive shapes. We propose a new low-level recognition and beautification system that can recognize eight primitive shapes, as well as combinations of these primitives, with recognition rates at 98.56%. Our system also automatically generates beautified versions of these shapes to provide feedback early in the sketching process. In addition to looking at geometric perception, much of our recognition success can be attributed to two new features, along with a new ranking algorithm, which have proven to be significant in distinguishing polylines from curved segments.

Sketch-based search and composition of 3D models

Abstract: There is growing interest in developing tools with which novice users can create detailed 3D models of their own designs. The most popular approaches to this problem include sketch-based interfaces and part-composition systems. The sketch-based modeling systems provide natural interfaces for creating 3D models from 2D sketches, but are generally limited to creating simple geometric models. The part-composition systems provide tools for combining parts extracted from a database of 3D models, and thus can generate very detailed 3D models, but usually with much higher overhead and expertise required by the user for manipulating 3D geometry interactively. In this paper, we introduce a new modeling method that overcomes these limitations by combining both approaches - we introduce a modeling system for parts composition with a sketching interface. The system allows the user to find a part in a database and composite it into a model with a single sketch. This approach combines the benefits of both approaches - i.e., it allows creation of highly detailed models/scenes (as details come from parts in the database), while 2D sketched strokes provide all the information for part selection and composition (no 3D manipulation is required, in general). To enable this modeling method, we investigate an algorithm for 3D shape search with 2D sketch as a shape query and a part placement algorithm which automatically orients, translates, scales, and attaches a new part into a modeling scene by taking the user sketch as a hint. User experiences with our prototype system show that novice users can create interesting and detailed models with our system.

Pattern Recognition Technologies and Applications: Recent Advances

Abstract: The nature of handwriting in our society has significantly altered over the ages due to the introduction of new technologies such as computers and the World Wide Web. With increases in the amount of signature verification needs, state of the art internet and paper-based automated recognition methods are necessary. "Pattern Recognition Technologies and Applications: Recent Advances" provides cutting-edge pattern recognition techniques and applications. Written by world-renowned experts in their field, this easy to understand book is a must have for those seeking explanation in topics such as on- and offline handwriting and speech recognition, signature verification, and gender classification.

New Frontiers in Artificial Intelligence

Abstract: Neg-Raising (NR) verbs form a class of verbs with a clausal complement that show the following behavior: when a negation syntactically attaches to the matrix predicate, it can semantically attach to the embedded predicate. This paper presents an account of NR predicates within Tree Adjoining Grammar (TAG). We propose a lexical semantic interpretation that heavily relies on a Montague-like semantics for TAG and on higher-order types.

Sentence recognition using artificial neural networks

Abstract: The paper describes an application of artificial neural networks (ANN) for natural language text reasoning. The task of knowledge discovery in text from a database, represented with a database file consisting of sentences with similar meanings but different lexico-grammatical patterns, was solved with ANNs which recognize the meaning of the text using training files with limited dictionary. The paper features recognition algorithms of text meaning from a selected source using 3-layer ANNs. Tests of the new method have also been described.

An agent-based framework for sketched symbol interpretation

Abstract: Recognizing hand-sketched symbols is a definitely complex problem. The input drawings are often intrinsically ambiguous, and require context to be interpreted in a correct way. Many existing sketch recognition systems avoid this problem by recognizing single segments or simple geometric shapes in a stroke. However, for a recognition system to be effective and precise, context must be exploited, and both the simplifications on the sketch features, and the constraints under which recognition may take place, must be reduced to the minimum. In this paper, we present an agent-based framework for sketched symbol interpretation that heavily exploits contextual information for ambiguity resolution. Agents manage the activity of low-level hand-drawn symbol recognizers, that may be heterogeneous for better adapting to the characteristics of each symbol to be recognized, and coordinate themselves in order to exchange contextual information, thus leading to an efficient and precise interpretation of sketches. We also present AgentSketch, a multi-domain sketch recognition system implemented according to the proposed framework. A first experimental evaluation has been performed on the domain of UML Use Case Diagrams to verify the effectiveness of the proposed approach.

Translation and scale-invariant gesture recognition in complex scenes

Abstract: Gestures are a natural means of communication between humans, and also a natural modality for human-computer interaction. Automatic recognition of gestures using computer vision is an important task in many real-world applications, such as sign language recognition, computer games control, virtual reality, intelligent homes, and assistive environments. In order for a gesture recognition system to be robust and deployable in non-laboratory settings, the system needs to be able to operate in complex scenes, with complicated backgrounds and multiple moving and skin-colored objects. In this paper we propose an approach for improving gesture recognition performance in such complex environments. The key idea is to integrate a face detection module into the gesture recognition system, and use the face location and size to make gesture recognition invariant to scale and translation. Our experiments demonstrate the significant advantages of the proposed method over alternative computer vision methods for gesture recognition.

Sketch-based educational games: "drawing" kids away from traditional interfaces

Abstract: Computer-based games and technologies can be significant aids for helping children learn. However, most computer-based games simply address the learning styles of visual and auditory learners. Sketch-based interfaces, however, can also address the needs of those children who learn better through tactile and kinesthetic approaches. Furthermore, sketch recognition can allow for automatic feedback to aid children without the explicit need for teacher to be present. In this paper, we present various sketch-based tools and games that promote tactile learning and entertainment for children.

Free-sketch recognition: putting the chi in sketching

Abstract: Sketch recognition techniques have generally fallen into two camps. Gesture-based techniques, such as those used by the Palm Pilot's Graffiti, can provide high-accuracy, but require the user to learn a particular drawing style in order for shapes to be recognized. Free-sketch recognition allows users to draw shapes as they would naturally, but most current techniques have low accuracies or require significant domain-level tweaking to make them usable. Our goal is to recognize free-hand sketches with high accuracy by developing generalized techniques that work for a variety of domains, including design and education. This is a work-in-progress, but we have made significant advancements toward our over-arching goal.

A Novel Pen-Based Flowchart Recognition System for Programming Teaching

Abstract: The electronic white board and the tablet PC are bringing new technologies to modern education. This paper presents a pen-based flowchart recognition system for programming teaching, which uses hybrid SVM-HMM algorithm for sketch recognition. In this algorithm, ICA is used to reduce the dimension of features, a set of fuzzy SVMs are used as preliminary feature classifiers to produce fix length feature vector, which acts as a probability evaluator in the hidden states of Hidden Markov Models, and HMMs are employed as finally classifiers to recognize the unknown pattern. Experiment results show the hybrid algorithm has good learning and recognition ability. And based on this algorithm, an intelligent whiteboard system for programming teaching is designed to identify the sketches into the programming flowchart, and finally converts it into C language programs. User's evaluation shows it is natural for the teachers and the students with a flexible and effective interactive teaching pattern. Therefore, such system brings a new programming teaching patterns and help students to stride the obstacle between the flowchart and the programming language. Students can learn the abstract programming idea and the concrete coding skills effectively and efficiently by the visual comparative learning assisted by the intelligent whiteboard system.

Spoken Digit Recognition using a Hierarchical Temporal Memory

Abstract: In this paper we explore the feasibility of the MemoryPrediction Theory, implemented in the form of a Hierarchical Temporal Memory (HTM), for automatic speech recognition. Up to nowHTMs have almost exclusively been applied to image processing. However, the underlying theory can also be used as an approach to active perception of audio signals. Using the software platform under development by NUMENTA we implemented a system for isolated digit recognition, the speech recognition task that can be most easily cast in a form similar to image recognition. Our results show that the HTM approach holds promises for speech recognition. At the same time it is clear that the present implementation is not ideally suited for processing signals that encode information mainly in dynamic changes.

STRAT: a Sketched-truss Recognition and Analysis Tool.

Abstract: The statically-determinate, pin-connected truss is a basic structural element used by engineers to create larger and more complex systems. Truss analysis and design are topics that virtually all students who study engineering mechanics are required to master, many of whom may experience difficulty with initial understanding. The mathematics used to analyze truss systems typically requires lengthy hand calculations or the assistance of proprietary computer-aided design (CAD) programs. To expedite work in this domain, we propose: STRAT (Sketched-Truss Recognition and Analysis Tool), a freehand sketch recognition system for solving truss problems. The STRAT system allows users to rapidly determine all of the unknown forces in a truss, using only a hand-drawn sketch of the truss itself. The focus of this article covers the design methodology and implementation of the STRAT system. Results from a preliminary user study are also presented.

Arabic handwriting recognition: Challenges and solutions

Abstract: Optical Characters Recognition (OCR) is one of the active subjects of research since the early days of computer science. Even if Arabic characters are used by more than a half a billion people; Arabic characters recognition has not received enough interests by the researchers. Little research progress has been achieved comparing to what has been done with Latin and Chinese. The cursive nature of the Arabic characters makes it more difficult to achieve a high accuracy in character recognition since even printed Arabic characters are in cursive form. This paper presents the main challenges (difficulties) researchers are facing and up to dated solutions (the common methods) are used for Arabic text recognition.

From paper to machine: extracting strokes from images for use in sketch recognition

Abstract: Sketching is a way of conveying ideas to people of diverse backgrounds and culture without any linguistic medium. With the advent of inexpensive tablet PCs, online sketches have become more common, allowing for stroke-based sketch recognition techniques, more powerful editing techniques, and automatic simulation of recognized diagrams. Online sketches provide significantly more information than paper sketches, but they still do not provide the flexibility, naturalness, and simplicity of a simple piece of paper. Recognition methods exist for paper sketches, but they tend to be domain specific and don't benefit from the advances of stroke-based sketch recognition. Our goal is to combine the power of stroke-based sketch recognition with the flexibility and ease of use of a piece of paper. In this paper we will present a stroke-tracing algorithm that can be used to extract stroke data from the pixilated image of the sketch drawn on paper. The presented method handles overlapping strokes and also attempts to capture sequencing information, which is helpful in many sketch recognition techniques. We present preliminary results of our algorithm on several paper-drawn, hand-sketched, scanned-in pixilated images.

SOUSA: sketch-based online user study applet

Abstract: Although existing domain-specific datasets are readily available, most sketch recognition researchers are forced to collect new data for their particular domain. Creating tools to collect and label sketched data can take time, and, if every researcher creates their own toolset, much time is wasted that could be better suited toward advanced research. Additionally, it is often the case that other researchers have performed collection studies and collected the same types of sketch data, resulting in large duplications of effort. We propose, and have built, a generalpurpose sketch collection and verification tool that allows researchers to design custom user studies through an online applet residing on our group's web page. By hosting such a tool through our site, we hope to provide researchers with a quick and easy way of collecting data. Additionally, our tool serves to create a universal repository of sketch data that can be made readily available to other sketch recognition researchers.

Human and Computer Evaluations of Face Sketches with Implications for Forensic Investigations

Abstract: Because sketches represent the original faces in a much concise yet recognizable form, they play an important role in criminal investigations, human perceptions and biometrics. In this work, we compared the performances of humans and a PCA-based algorithm in recognizing face sketches. A total of 250 sketches of 50 subjects were involved. All sketches were drawn manually by five artists (each artist drew 50 sketches, one for each subject). Experiments were carried out by matching sketches in a probe set to photos in a gallery set. This study resulted in the following findings: (i) A large inter-artist variation in sketch recognition rate was observed; (ii) Fusing sketches from different artists significantly improved the performance; (iii) Human performance seems correlated with that of the algorithm; (iv) The algorithm was superior with sketches of less distinctive features, while humans used tonality (or pigmentation) cues more efficiently.

A data collection tool for sketched diagrams

Abstract: Repositories of digital ink sketches would be invaluable for testing and evaluation of sketch recognition software. However, there is no existing tool for flexible data collection and management of digital ink data for building repositories of hand drawn diagrams. We present a tool for the efficient collection, management and analysis of ink data. A resultant dataset records each ink stroke accompanied by participant and diagram information, stroke labels and measurements of various stroke features. This tool enables the effective construction of a large database of sketches to aid the development of recognition techniques.

Real-time object recognition using local features on a DSP-based embedded system

Abstract: In the last few years, object recognition has become one of the most popular tasks in computer vision. In particular, this was driven by the development of new powerful algorithms for local appearance based object recognition. So-called “smart cameras” with enough power for decentralized image processing became more and more popular for all kinds of tasks, especially in the field of surveillance. Recognition is a very important tool as the robust recognition of suspicious vehicles, persons or objects is a matter of public safety. This simply makes the deployment of recognition capabilities on embedded platforms necessary. In our work we investigate the task of object recognition based on state-of-the-art algorithms in the context of a DSP-based embedded system. We implement several powerful algorithms for object recognition, namely an interest point detector together with an region descriptor, and build a medium-sized object database based on a vocabulary tree, which is suitable for our dedicated hardware setup. We carefully investigate the parameters of the algorithm with respect to the performance on the embedded platform. We show that state-of-the-art object recognition algorithms can be successfully deployed on nowadays smart cameras, even with strictly limited computational and memory resources.

GLADDER: combining gesture and geometric sketch recognition

Abstract: Sketch recognition systems usually recognize strokes either as stylistic gestures or geometric shapes. Both techniques have their advantages. This paper presents a method for integrating gesture-based and geometric recognition techniques, significantly outperforming either technique on its own.

Integration of Face and Hand Gesture Recognition

Abstract: Face recognition and hand gesture recognition technologies have been developed separately for many years. Usually they are treated as independent systems. In this paper, we integrate the face and hand gesture recognition. We claim that the face recognition rate can be improved by hand gesture recognition. Also, we propose a security elevator scenario. Finally, we simulate this security elevator scenario by PCA method, based on the ORL database, and show that the face recognition rate and overall accuracy is improved after integration. We believe that this is a general method to integrate two recognition engines, not only for face and hand gesture recognition.

Using a geometric-based sketch recognition approach to sketch Chinese radicals

Abstract: Unlike English, where unfamiliar words can be queried for its meaning by typing out its letters, the analogous operation in Chinese is far from trivial due to the nature of its written language. One approach for querying Chinese characters involve referencing their dictionary component called radicals. This is advantageous since users would not need to know their pronunciation nor their stroke-order, a requirement in other querying approaches. Currently though, sketching a character’s radical for querying is an unsupported capability in existing systems. Using the geometric-based LADDER sketching language combined with the Sezgin lowlevel recognizer, we were able to construct an application which can first recognize handwritten sketches of Chinese radical, and then output candidate Chinese characters which contain that radical. Thus, we were able to demonstrate that a geometric-based sketch recognition approach can be used to easily build applications for recognizing symbols related to Chinese characters while having reasonable recognition rates. Unlike current image-based recognition systems, our system also maintains stroke order information of characters. Since stroke order is important in written Chinese, our system can be easily expanded for use in Chinese language education by providing visual feedback to students on correct stroke order.

Technologies of Hand Gesture Recognition Based on Vision

Abstract: Computer has been widely used in all parts of people's daily life.The study of the mutual relationship between people and computer has also become the focus of the scientific research.The ability for computer to visually recognize hand gestures is essential for future human-computer interaction.Moreover hand gesture recognition based on vision can promote the sign language recognition development.Sign language recognition can eliminate the exchange barrier between healthy person and deaf-mute,which enables them to obtain the healthy person's normal life,helps them to take part in social activity.Presents the development,technical difficulty and elaborates the system principle and the composition of hand gestures recognition based on vision.Moreover,introduced the modeling of hand gesture and the technique of hand gestures recognition.

Recognition of books by verification and retraining

Abstract: The problem of character recognition in a book should be formulated significantly different from that of a single page or word. An ideal approach to design such a recognizer is to adapt the classifier to the font and style of the collection. In this paper, we propose an adaptation framework to recognize characters in a book with a learning framework. In the proposed system, the post processor verifies the output of the recognition module, which is further used for learning and thus to improve the performance over iteration. Experiments are conducted on about 500,000 annotated symbols from five books in Malayalam (an Indian language). We achieve an average improvement of 14% in classification accuracy.

A Geometric-based Sketch Recognition Approach for Handwritten Mandarin Phonetic Symbols I.

Abstract: Inputting written Chinese, unlike written English, is a non-trivial operation using a standard keyboard. To accommodate this operation, numerous existing phonetic systems using the Roman alphabet were adopted as a means of input while still making use of a Western keyboard. With the growing prevalence of computing devices capable of pen-based input, naturally sketching written Chinese using a phonetic system becomes possible, and is also generally faster and simpler than sketching entire Chinese characters. One method for sketching Chinese characters for computing devices capable of pen-based input involves using an existing non-alphabetic phonetic system called the Mandarin Phonetic Symbols I (MPS1). The benefits of inputting Chinese characters by its corresponding MPS1 symbols – unlike letters from its alphabetic-based counterpart – is that it retains the phonemic components of the corresponding Chinese characters. The work in the paper describes our geometric-based MPS1 recognition system, a system designed particularly for novice users of MPS1 symbols that gives reasonable vision-based recognition rates and provides useful feedback for symbols drawn with incorrect sketching technique such as stroke order.

MARQS: retrieving sketches learned from a single example using a dual-classifier

Abstract: Mouse and keyboard interfaces handle traditional text-based queries, and standard search engines provide for effective text-based search. However, everyday documents are filled with not only text, but photos, cartoons, diagrams, and sketches. These images can often be easier to recall than the surrounding text. In an effort to make human computer interaction handle more forms of human-human interaction, sketching has recently become an important means of interacting with computer systems. We propose extending the traditional monomodal model of text-based search to include the capabilities of sketch-based search. Our goal is to create a sketch-based search that can find documents from a single query sketch. We imagine an important use for this technology would be to allow users to search a computerized laboratory notebook for a previously drawn sketch. Because such as sketch will have initially been drawn only a single time, it is important that the search-by-sketch system (1) recognize a wide range of shapes that are not necessarily geometric nor drawn in the same way each time, (2) recognize a query example from only one initial training example, and (3) learn from successful queries to improve accuracy over time. We present here such an algorithm. To test the algorithm, we implemented a proof-of-concept-system: MARQS, a system that uses sketches to query existing media albums. Preliminary results show that the system yielded an average search rank of 1.51, indicating that the correct sketch is presented as either the top or second search result on average.

Gesture Recognition Based on Manifold Learning

Abstract: Current feature-based gesture recognition systems use human-chosen features to perform recognition. Effective features for classification can also be automatically learned and chosen by the computer. In other recognition domains, such as face recognition, manifold learning methods have been found to be good nonlinear feature extractors. Few manifold learning algorithms, however, have been applied to gesture recognition. Current manifold learning techniques focus only on spatial information, making them undesirable for use in the domain of gesture recognition where stroke timing data can provide helpful insight into the recognition of hand-drawn symbols. In this paper, we develop a new algorithm for multi-stroke gesture recognition, which integrates timing data into a manifold learning algorithm based on a kernel Isomap. Experimental results show it to perform better than traditional human-chosen feature-based systems.