Topic
Sketch recognition
About: Sketch recognition is a research topic. Over the lifetime, 1611 publications have been published within this topic receiving 40284 citations.
Papers published on a yearly basis
Papers
More filters
01 Jan 2022
TL;DR: In this article , the authors propose a primitive-based sketch abstraction task, which maps each stroke of a sketch to its most similar primitive in a given set, predicting an affine transformation that aligns the selected primitive to the target stroke.
Abstract: Humans show high-level of abstraction capabilities in games that require quickly communicating object information. They decompose the message content into multiple parts and communicate them in an interpretable protocol. Toward equipping machines with such capabilities, we propose the Primitive-based Sketch Abstraction task where the goal is to represent sketches using a fixed set of drawing primitives under the influence of a budget. To solve this task, our Primitive-Matching Network (PMN), learns interpretable abstractions of a sketch in a self supervised manner. Specifically, PMN maps each stroke of a sketch to its most similar primitive in a given set, predicting an affine transformation that aligns the selected primitive to the target stroke. We learn this stroke-to-primitive mapping end-to-end with a distance-transform loss that is minimal when the original sketch is precisely reconstructed with the predicted primitives. Our PMN abstraction empirically achieves the highest performance on sketch recognition and sketch-based image retrieval given a communication budget, while at the same time being highly interpretable. This opens up new possibilities for sketch analysis, such as comparing sketches by extracting the most relevant primitives that define an object category. Code is available at https://github.com/ExplainableML/sketch-primitives .
04 May 2015
TL;DR: An online shape recognizer that identifies multi-stroke geometric shapes without a plan library is presented, Inspired by mirroring processes hypothesized to take place in socially-intelligent brains, which uses a shape-drawing planner for drawn-shape recognition.
Abstract: Humans increasingly use sketches, drawn on paper, on a computer, or via hand gestures in the air, as part of their communications with agents, robots, and other humans. To recognize shapes in sketches, most existing work focuses on offline (post-drawing) recognition methods, trained on large sets of examples . Given the infinite number of ways in which shapes can appear-rotated, scaled, translated-and given inherent inaccuracies in the drawings, these methods do not allow on-line recognition, and require a very large library (or expensive pre-processing) in order to recognize even a small number of shapes. We present an online shape recognizer that identifies multi-stroke geometric shapes without a plan library. Inspired by mirroring processes hypothesized to take place in socially-intelligent brains, the recognizer uses a shape-drawing planner for drawn-shape recognition. It is a form of plan recognition from planning.
TL;DR: The proposed method first synthesizes pseudo sketches by computing the locality sensitive histogram and dense illumination invariant features from the resized face photos, then extracts discriminative features by computing histogram of averaged oriented gradients on the query sketch and pseudo sketches, and finally find a match with the shortest cosine distance in the feature space.
Abstract: This letter proposes a new face sketch recognition method. Given a query sketch and face photos in a database, the proposed method first synthesizes pseudo sketches by computing the locality sensitive histogram and dense illumination invariant features from the resized face photos, then extracts discriminative features by computing histogram of averaged oriented gradients on the query sketch and pseudo sketches, and finally find a match with the shortest cosine distance in the feature space. It achieves accuracy comparable to the state-of-the-art while showing much more robustness than the existing face sketch recognition methods. key words: face sketch recognition, photo-sketch matching, locality sensitive histogram, histogram of averaged oriented gradients
01 Jan 2004
TL;DR: The goal was to check if there were ways of filling in image features that minimized the number of partial interpretations created by the time the object is completely drawn.
Abstract: Why: In online sketching, strokes are put on the sketching surface one at a time We propose a sketch recognition strategy that takes this incremental nature of the sketching process into account Current approaches to online sketch recognition apply object recognition methods developed for static images to sketches that are formed in a dynamic, incremental fashion The incremental nature of sketching necessitates a strategy for assigning image features to model features 1 that keeps the number of ambiguous interpretations minimum, or alternatively, the branching factor of the corresponding interpretation tree low 2 Otherwise, creating all plausible interpretations that can be generated at a given time may result in too many partial interpretations How: The approach we take is to analyze object models and derive a recognition strategy specifying the actions to take under different drawing scenarios The actions that a recognizer can take may include: Checking if certain constraints are satisfied between image features Creating a partial interpretation by assigning an image feature to a model feature Delaying the partial interpretation creation until more strokes are drawn by the user The kinds of analysis include inspection of object descriptions, as well as simulations with hand drawn instances of the object We conducted a simulation with a hand drawn example to serve as a proof of concept experiment Our goal was to check if there were ways of filling in image features that minimized the number of partial interpretations created by the time the object is completely drawn The recognition strategy used in our experiment is a variant of the interpretation tree algorithm After each stroke is drawn, for each type of object to be recognized: If there are no partial interpretations from the object class we are trying to recognize, and if the geometric primitive derived from the latest stroke fits into a model feature without violating any constraints, create a new partial interpretation with that primitive assigned to the model feature If there are existing partial interpretations that can be extended with the latest primitive without violating any constraints, these interpretations are cloned and extended We implemented a stick-figure recognizer using this strategy We recorded raw strokes for a stick