Sketch recognition

About: Sketch recognition is a research topic. Over the lifetime, 1611 publications have been published within this topic receiving 40284 citations.

Evaluation of a natural sketch interface for truss FBDs and analysis

Abstract: Mechanix is a sketch recognition tool that provides an efficient means for engineering students to learn how to draw truss free-body diagrams (FBDs) and solve truss problems. The system allows students to sketch these FBDs into a tablet computer or by using a mouse just as they would by hand, a mouse can also be used to draw the sketch using a regular computer and monitor. Mechanix is then able to provide immediate feedback to the students and tell them if they are missing any components of the FBD, without providing answers. The program is also able to tell them whether their solved reaction or member forces are correct or not. This paper presents a study to evaluate the effectiveness and advantages of using Mechanix in the classroom, as a supplement to traditional teaching and learning methods. Current results demonstrate that students believe Mechanix enhances their learning and are highly engaged when using it. Future work on the refinement of the program is also discussed.

Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesis

Abstract: In this paper, we explore open-domain sketch-to-photo translation, which aims to synthesize a realistic photo from a freehand sketch with its class label, even if the sketches of that class are missing in the training data. It is challenging due to the lack of training supervision and the large geometric distortion between the freehand sketch and photo domains. To synthesize the absent freehand sketches from photos, we propose a framework that jointly learns sketch-to-photo and photo-to-sketch generation. However, the generator trained from fake sketches might lead to unsatisfying results when dealing with sketches of missing classes, due to the domain gap between synthesized sketches and real ones. To alleviate this issue, we further propose a simple yet effective open-domain sampling and optimization strategy to "fool" the generator into treating fake sketches as real ones. Our method takes advantage of the learned sketch-to-photo and photo-to-sketch mapping of in-domain data and generalizes it to the open-domain classes. We validate our method on the Scribble and SketchyCOCO datasets. Compared with the recent competing methods, our approach shows impressive results in synthesizing realistic color, texture, and maintaining the geometric composition for various categories of open-domain sketches.

An integrated multimedia environment for speech recognition using handwriting and written gestures

Abstract: To provide a natural interface to the computer, we present an integrated speech, gesture, and handwriting recognition system. By integrating these technologies, we can easily accomplish all the tasks that are needed to use computers and their applications, including many tasks which neither technology can adequately perform by itself. This integrated environment not only makes it easier for novices to use computers but also increases the productivity of experienced computer users.

Synthetic data generation of time series data

Abstract: A method of generating synthetic data from time series data, such as from handwritten characters, words, sentences, mathematics, and sketches that are drawn with a stylus on an interactive display or with a finger on a touch device. This computationally efficient method is able to generate realistic variations of a given sample. In a handwriting or sketch recognition context, synthetic data is generated from real data in order to train recognizers and thus improve recognition accuracy when only a limited number of samples are available. Similarly, synthetic data can also be used to test and validate such recognizers. Also discussed is a dynamic time warping based approach for both segmented and continuous data that is designed to be a robust, go-to method for gesture recognition across a variety of modalities using only limited training samples.

Three dimensional gesture recognition using modified matching algorithm

Abstract: User-friendly Human-Computer interaction becomes more important accordance with rapid development of various information systems. In this paper we describe a three-dimensional gesture recognition algorithm and a system that adopts the algorithm for non-contact human-computer interaction. From sequence of stereo images, five feature regions are extracted with simple color segmentation algorithm and then those are used for three dimensional locus calculation processing. However, the result is not so stable, noisy, that we introduce principal component analysis method to get more robust gesture recognition results. This method can overcome the weakness of conventional algorithms since it directly uses three-dimensional information for human gesture recognition.