Sketch recognition

About: Sketch recognition is a research topic. Over the lifetime, 1611 publications have been published within this topic receiving 40284 citations.

BopoNoto: An Intelligent Sketch Education Application for Learning Zhuyin Phonetic Script.

Abstract: The zhuyin phonetic script not only provides greater learning benefits compared to romanization systems for Chinese Mandarin language students with existing English fluency, but also allows students to take advantage of its use in Taiwan where it exists in both official and popular usage. However, while penenabled computing educational applications can assist traditional pedagogical approaches for accelerating mastery of the script, existing approaches are either constrained in providing writing assessment, catered to native language users, or are less flexible in recognizing more natural writing. We present BopoNoto, an intelligent sketch education application for language students to learn zhuyin. Our application provides a sketching interface for practicing the symbols, and a sketch recognition system for assessing the visual and technical correctness of their input. From our evaluations, BopoNoto successfully demonstrates strong results in understanding and assessing students’ written input. Keywords-sketch recognition; intelligent user interface; intelligent tutoring system; Chinese Mandarin; zhuyin; bopomofo

Combined hand gesture--speech model for human action recognition.

Abstract: This study proposes a dynamic hand gesture detection technology to effectively detect dynamic hand gesture areas, and a hand gesture recognition technology to improve the dynamic hand gesture recognition rate. Meanwhile, the corresponding relationship between state sequences in hand gesture and speech models is considered by integrating speech recognition technology with a multimodal model, thus improving the accuracy of human behavior recognition. The experimental results proved that the proposed method can effectively improve human behavior recognition accuracy and the feasibility of system applications. Experimental results verified that the multimodal gesture-speech model provided superior accuracy when compared to the single modal versions.

Face Recognition by Expression-Driven Sketch Graph Matching

Abstract: We present a novel face recognition method using automatically extracted sketch by a multi-layer grammatical face model. First, the observed face is parsed into a 3- layer (face, parts and sketch) graph. In the sketch layer, the nodes not only capture the local features (strength, orientation and profile of the edge), but also remember the global information inherited from the upper layers (i.e. the facial part they belong to and status of the part). Next, a sketch graph matching is performed between the parsed graph and a pre-built reference graph database, in which each individual has a parsed sketch graph. Similar to the other successful edge-based methods in the literature, the use of sketch increases the robustness of recognition under varying lighting conditions. Furthermore, with high-level semantic understanding of the face, we are able to perform an intelligent recognition process driven by the status of the face, i.e. changes in expressions and poses. As shown in the experiment, our method overcomes the significant drop in accuracy under expression changes suffered by other edge-based methods.

Sketching and computer-aided conceptual design

Abstract: Sketching is widely considered to be an essential activity during conceptual design, and many argue that CAD tools should be faithful to the sketching metaphor for conceptual design. However, CAD tools have progressed significantly in recent years, and there is growing experimental evidence that existing CAD tools can be as effective as sketching. Recent research in cognitive psychology supports the idea that the sketching metaphor is not necessarily ideal, and that a 3D geometric modeling metaphor might better support human cognitive processes. Informal experiments in CAD modeling of sample geometric shapes reported in the sketch recognition literature shows that the two approaches are comparable. This evidence suggests that computer sketch recognition may be unnecessary, and that efforts should be directed toward improving the human factors aspects of current CAD software to better support the needs of conceptual design.

A Hierarchical Human Activity Recognition Framework Based on Automated Reasoning

Abstract: Conventional human activity recognition approaches are mainly based on machine learning methods, which are not working well for composite activity recognition due to the complexity and uncertainty of real scenarios. We propose in this paper an automated reasoning based hierarchical framework for human activity recognition. This approach constructs a hierarchical structure for representing the composite activity by a composition of lower-level actions and gestures according to its semantic meaning. This hierarchical structure is then transformed into logical formulas and rules, based on which the resolution based automated reasoning is applied to recognize the composite activity given the recognized lower-level actions by machine learning methods.