Showing papers on "Sketch published in 2011"

What is the self

Abstract: In this paper I briefly sketch a theory that answers the question "what is the self?," where this question is understood in a scientific sense that includes both natural science and systematic fundamental metaphysics. As selves, we are essentially rational human minded animals or real persons in a fully natural and desperately non-ideal world-animals with meaningful lives, for better or worse.

Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors

Abstract: We introduce a benchmark for evaluating the performance of large-scale sketch-based image retrieval systems. The necessary data are acquired in a controlled user study where subjects rate how well given sketch/image pairs match. We suggest how to use the data for evaluating the performance of sketch-based image retrieval systems. The benchmark data as well as the large image database are made publicly available for further studies of this type. Furthermore, we develop new descriptors based on the bag-of-features approach and use the benchmark to demonstrate that they significantly outperform other descriptors in the literature.

Edgel index for large-scale sketch-based image search

Abstract: Retrieving images to match with a hand-drawn sketch query is a highly desired feature, especially with the popularity of devices with touch screens. Although query-by-sketch has been extensively studied since 1990s, it is still very challenging to build a real-time sketch-based image search engine on a large-scale database due to the lack of effective and efficient matching/indexing solutions. The explosive growth of web images and the phenomenal success of search techniques have encouraged us to revisit this problem and target at solving the problem of web-scale sketch-based image retrieval. In this work, a novel index structure and the corresponding raw contour-based matching algorithm are proposed to calculate the similarity between a sketch query and natural images, and make sketch-based image retrieval scalable to millions of images. The proposed solution simultaneously considers storage cost, retrieval accuracy, and efficiency, based on which we have developed a real-time sketch-based image search engine by indexing more than 2 million images. Extensive experiments on various retrieval tasks (basic shape search, specific image search, and similar image search) show better accuracy and efficiency than state-of-the-art methods.

CogSketch: sketch understanding for cognitive science research and for education.

Abstract: Sketching is a powerful means of working out and communicating ideas. Sketch understanding involves a combination of visual, spatial, and conceptual knowledge and reasoning, which makes it both challenging to model and potentially illuminating for cognitive science. This paper describes CogSketch, an ongoing effort of the NSF-funded Spatial Intelligence and Learning Center, which is being developed both as a research instrument for cognitive science and as a platform for sketch-based educational software. We describe the idea of open-domain sketch understanding, the scientific hypotheses underlying CogSketch, and provide an overview of the models it employs, illustrated by simulation studies and ongoing experiments in creating sketch-based educational software.

Sketch-sketch revolution: an engaging tutorial system for guided sketching and application learning

Abstract: We describe Sketch-Sketch Revolution, a new tutorial system that allows any user to experience the success of drawing content previously created by an expert artist Sketch-Sketch Revolution not only guides users through the application user interface, it also provides assistance with the actual sketching In addition, the system offers an authoring tool that enables artists to create content and then automatically generates a tutorial from their recorded workflow history Sketch-Sketch Revolution is a unique hybrid tutorial system that combines in-product, content-centric and reactive tutorial methods to provide an engaging learning experience A qualitative user study showed that our system successfully taught users how to interact with a drawing application user interface, gave users confidence they could recreate expert content, and was uniformly considered useful and easy to use

Photosketcher: Interactive Sketch-Based Image Synthesis

Abstract: Photosketcher is an interactive system for progressively synthesizing novel images using only sparse user sketches as input. Photosketcher works on the image content exclusively; it doesn't require keywords or other metadata associated with the images. Users sketch the rough shape of a desired image part, and Photosketcher searches a large collection of images for it. The search is based on a bag-of-features approach that uses local descriptors for translation-invariant retrieval of image parts. Composition is based on user scribbles: from the scribbles, Photosketcher predicts the desired part using Gaussian mixture models and computes an optimal seam using graph cuts. To further reduce visible seams, users can blend the composite image in the gradient domain.

ChemInk: a natural real-time recognition system for chemical drawings

Abstract: We describe a new sketch recognition framework for chemical structure drawings that combines multiple levels of visual features using a jointly trained conditional random field. This joint model of appearance at different levels of detail makes our framework less sensitive to noise and drawing variations, improving accuracy and robustness. In addition, we present a novel learning-based approach to corner detection that achieves nearly perfect accuracy in our domain. The result is a recognizer that is better able to handle the wide range of drawing styles found in messy freehand sketches. Our system handles both graphics and text, producing a complete molecular structure as output. It works in real time, providing visual feedback about the recognition progress. On a dataset of chemical drawings our system achieved an accuracy rate of 97.4%, an improvement over the best reported results in literature. A preliminary user study also showed that participants were on average over twice as fast using our sketch-based system compared to ChemDraw, a popular CAD-based tool for authoring chemical diagrams. This was the case even though most of the users had years of experience using ChemDraw and little or no experience using Tablet PCs.

gSketch: On Query Estimation in Graph Streams

Abstract: Many dynamic applications are built upon large network infrastructures, such as social networks, communication networks, biological networks and the Web. Such applications create data that can be naturally modeled as graph streams, in which edges of the underlying graph are received and updated sequentially in a form of a stream. It is often necessary and important to summarize the behavior of graph streams in order to enable effective query processing. However, the sheer size and dynamic nature of graph streams present an enormous challenge to existing graph management techniques. In this paper, we propose a new graph sketch method, gSketch, which combines well studied synopses for traditional data streams with a sketch partitioning technique, to estimate and optimize the responses to basic queries on graph streams. We consider two different scenarios for query estimation: (1) A graph stream sample is available; (2) Both a graph stream sample and a query workload sample are available. Algorithms for different scenarios are designed respectively by partitioning a global sketch to a group of localized sketches in order to optimize the query estimation accuracy. We perform extensive experimental studies on both real and synthetic data sets and demonstrate the power and robustness of gSketch in comparison with the state-of-the-art global sketch method.

gSketch: on query estimation in graph streams

Abstract: Many dynamic applications are built upon large network infrastructures, such as social networks, communication networks, biological networks and the Web. Such applications create data that can be naturally modeled as graph streams, in which edges of the underlying graph are received and updated sequentially in a form of a stream. It is often necessary and important to summarize the behavior of graph streams in order to enable effective query processing. However, the sheer size and dynamic nature of graph streams present an enormous challenge to existing graph management techniques. In this paper, we propose a new graph sketch method, gSketch, which combines well studied synopses for traditional data streams with a sketch partitioning technique, to estimate and optimize the responses to basic queries on graph streams. We consider two different scenarios for query estimation: (1) A graph stream sample is available; (2) Both a graph stream sample and a query workload sample are available. Algorithms for different scenarios are designed respectively by partitioning a global sketch to a group of localized sketches in order to optimize the query estimation accuracy. We perform extensive experimental studies on both real and synthetic data sets and demonstrate the power and robustness of gSketch in comparison with the state-of-the-art global sketch method.

The lived body in design: mapping the terrain

Abstract: We briefly sketch an overview of emerging design research and practice, which values the lived body as a central theoretical foundation in the design of interactive technologies. Three main areas of research activity are presented: theoretical and philosophical perspectives on bodies and embodiment; concepts of the body; and design approaches and methods for working with the body and bodily literacy.

Data analysis on interactive whiteboards through sketch-based interaction

Abstract: When faced with the task of understanding complex data, it is common for people to work on whiteboards, where they can collaborate with others, brainstorm lists of important questions, and sketch simple visualizations. However, these sketched visualizations seldom contain real data. We address this gap by extending these sketched whiteboard visualizations with the actual data to be analyzed. Guided by an iterative design process, we developed a better understanding of the challenges involved in bringing sketch-based interaction to data analysis. In this work we contribute insights into the design challenges of sketch-based charting, and we present SketchVis, a system that leverages hand-drawn input for exploring data through simple charts.

A Data Streaming Method for Monitoring Host Connection Degrees of High-Speed Links

Abstract: Due to the massive amount of data in high-speed network traffic and the limit on processing capability, it is a great challenge to accurately measure and monitor network traffic over high-speed links online. A new data structure is presented in this paper for locating the hosts associated with large connection degrees or significant changes in connection degrees based on the reversible connection degree sketch to monitor anomalous network traffic. The reversible connection degree sketch builds a compact summary of host connection degrees efficiently and accurately. For each packet coming, it only needs to set several bits selected in a bit array by a group of hash functions. These hash functions are designed based on the Chinese Remainder Theorem so that the in-degree or out-degree associated with a given host can be accurately estimated. With this new data structure, we develop a new reverse sketch method for locating abnormal hosts. Although the reversible connection degree sketch does not preserve any host address information, we can analytically reconstruct the host addresses associated with large connection degrees or significant changes in connection degrees by a simple calculation purely based on the characteristics of the hash functions. Furthermore, a reinforced reversible connection degree sketch, the double connection degree sketch, is developed to reduce false positives which are commonly encountered in the sketch-based methods. A traffic monitoring system based on this double connection degree is developed to detect and classify the abnormal hosts associated with large connection degrees or significant changes in connection degrees. The experiments are conducted based on the actual network traffic and the testing results show that our method is accurate and efficient.

Integration of sketch-based interaction and computer data analysis

Abstract: Architecture that integrates the benefits of natural user interaction such as freeform sketch with computer-aided charting. The architecture integrates natural user interaction utilizing multiple modalities (e.g., sketch, multi-touch, etc.) with computer supported data analysis that allows users to explore data by drawing charts using simple strokes. Natural user interactions can be utilized to change chart types by drawing symbols, transform data by applying functions, filter data by drawing strikethrough on legends, etc. Additionally, the architecture makes an inference of visualizations the user intended from user-drawn strokes, such as the axes of a graph, the words of a label, etc. When appropriate, the architecture automatically completes visualizations.

NaturaSketch: Modeling from Images and Natural Sketches

Abstract: Sketching on paper is a quick and easy way to communicate ideas. However, many sketch-based systems require people to draw in contrived ways instead of sketching freely as they would on paper. NaturaSketch affords a more natural interface through multiple strokes that overlap, cross, and connect. It also features a meshing algorithm to support multiple strokes of different classifications, which lets users design complex 3D shapes from sketches drawn over existing images. To provide a familiar workflow for object design, a set of sketch annotations can also specify modeling and editing operations. NaturaSketch empowers designers to produce a variety of models quickly and easily.

Face Sketch-Photo Synthesis under Multi-dictionary Sparse Representation Framework

Abstract: Sketch-photo synthesis is one of the important research issues of heterogeneous image transformation. Some available popular synthesis methods, like locally linear embedding (LLE), usually generate sketches or photos with lower definition and blurred details, which reduces the visual quality and the recognition rate across the heterogeneous images. In order to improve the quality of the synthesized images, a multi-dictionary sparse representation based face sketch-photo synthesis model is constructed. In the proposed model, LLE is used to estimate an initial sketch or photo, while the multi-dictionary sparse representation model is applied to generate the high frequency and detail information. Finally, by linear superimposing, the enhanced face sketch or photo can be obtained. Experimental results show that sketches and photos synthesized by the proposed method have higher definition and much richer detail information resulting in a higher face recognition rate between sketches and photos.

Indexing and Mathematical Explanation

Abstract: We discuss a recent attempt by Chris Daly and Simon Langford to do away with mathematical explanations of physical phenomena. Daly and Langford suggest that mathematics merely indexes parts of the physical world, and on this understanding of the role of mathematics in science, there is no need to countenance mathematical explanation of physical facts. We argue that their strategy is at best a sketch and only looks plausible in simple cases. We also draw attention to how frequently Daly and Langford find themselves in conflict with mathematical and scientific practice.

Sketch-based Dynamic Illustration of Fluid Systems

Abstract: This paper presents a lightweight sketching system that enables interactive illustration of complex fluid systems. Users can sketch on a 2.5-dimensional (2.5D) canvas to design the shapes and connections of a fluid circuit. These input sketches are automatically analyzed and abstracted into a hydraulic graph, and a new hybrid fluid model is used in the background to enhance the illustrations. The system provides rich simple operations for users to edit the fluid system incrementally, and the new internal flow patterns can be simulated in real time. Our system is used to illustrate various fluid systems in medicine, biology, and engineering. We asked professional medical doctors to try our system and obtained positive feedback from them.

Sketch4match — Content-based image retrieval system using sketches

Abstract: The content based image retrieval (CBIR) is one of the most popular, rising research areas of the digital image processing. Most of the available image search tools, such as Google Images and Yahoo! Image search, are based on textual annotation of images. In these tools, images are manually annotated with keywords and then retrieved using text-based search methods. The performances of these systems are not satisfactory. The goal of CBIR is to extract visual content of an image automatically, like color, texture, or shape. This paper aims to introduce the problems and challenges concerned with the design and the creation of CBIR systems, which is based on a free hand sketch (Sketch based image retrieval — SBIR). With the help of the existing methods, describe a possible solution how to design and implement a task spesific descriptor, which can handle the informational gap between a sketch and a colored image, making an opportunity for the efficient search hereby. The used descriptor is constructed after such special sequence of preprocessing steps that the transformed full color image and the sketch can be compared. We have studied EHD, HOG and SIFT. Experimental results on two sample databases showed good results. Overall, the results show that the sketch based system allows users an intuitive access to search-tools. The SBIR technology can be used in several applications such as digital libraries, crime prevention, photo sharing sites. Such a system has great value in apprehending suspects and indentifying victims in forensics and law enforcement. A possible application is matching a forensic sketch to a gallery of mug shot images. The area of retrieve images based on the visual content of the query picture intensified recently, which demands on the quite wide methodology spectrum on the area of the image processing.

Primal Sketch: Integrating Texture and Structure

Abstract: Following Marr's insight, we propose a generative image representation called primal sketch, which integrates two modeling components. The first component explains the structural part of an image, such as object boundaries, by a hidden layer of image primitives. The second component models the remaining textural part without distinguishable elements by Markov random fields that interpolate the structural part of the image. We adopt an artist's notion by calling the two components sketchable and non-sketchable parts respectively. A dictionary of image primitives are used for modeling structures in natural images, and each primitive is specified by variables for its photometric, geometric, and topological attributes. The primitives in the image representation are not independent but organized in an sketch graph. This sketch graph is modeled by a spatial Markov model that enforces Gestalt organizations. The inference of the sketch graph consists of two phases. Phase I sequentially adds the most prominent image primitives in a procedure similar to matching pursuit. Phase II edits the sketch graph by a number of graph operators to achieve good Gestalt organizations. Experiments show that the primal sketch model produces satisfactory results for a large number of generic images. The primal sketch model is not only a parsimonious image representation for lossy image coding, but also provides a meaningful mid-level generic representation for other vision tasks.

Sketch-based image search

Abstract: Sketch-based image search may include receiving a query curve as a sketch query input and identifying a first plurality of oriented points based on the query curve. The first plurality of oriented points may be used to locate at least one image having a curve that includes a second plurality of oriented points that match at least some of the first plurality of oriented points. Implementations also include indexing a plurality of images by identifying at least one curve in each image and generating an index comprising a plurality of oriented points as index entries. The index entries are associated with the plurality of images based on corresponding oriented points in the identified curves in the images.

Grammar-based techniques for creating ground-truthed sketch corpora

Abstract: Although publicly available, ground-truthed corpora have proven useful for training, evaluating, and comparing recognition systems in many domains, the availability of such corpora for sketch recognizers, and math recognizers in particular, is currently quite poor. This paper presents a general approach to creating large, ground-truthed corpora for structured sketch domains such as mathematics. In the approach, random sketch templates are generated automatically using a grammar model of the sketch domain. These templates are transcribed manually, then automatically annotated with ground-truth. The annotation procedure uses the generated sketch templates to find a matching between transcribed and generated symbols. A large, ground-truthed corpus of handwritten mathematical expressions presented in the paper illustrates the utility of the approach.

An Analysis of the Blind Variation and Selective Retention Theory of Creativity

Abstract: Picasso's Guernica sketches continue to provide a fruitful testing ground for examining and assessing the Blind Variation Selective Retention (BVSR) theory of creativity. Nonmonotonicity—e.g., as indicated by a lack of similarity of successive sketches—is not evidence of a selectionist process. Although the notion of blindness originally implied randomness, it now encompasses phenomena that bias idea generation, e.g., the influence of remote associations on sketch ideas. However, for a selectionist framework is to be applicable, such biases must be negligible, otherwise evolutionary change is attributed to those biases, not to selection. The notion of variants should not be applied to creativity; without a mechanism of inheritance, there is no basis upon which to delineate, for example, which sketch ideas are or are not variants of a given sketch idea. The notion of selective retention is also problematic. Selection provides an explanation when acquired change is not transmitted; it cannot apply to Picass...

Approximate Scalable Bounded Space Sketch for Large Data NLP

Abstract: We exploit sketch techniques, especially the Count-Min sketch, a memory, and time efficient framework which approximates the frequency of a word pair in the corpus without explicitly storing the word pair itself. These methods use hashing to deal with massive amounts of streaming text. We apply Count-Min sketch to approximate word pair counts and exhibit their effectiveness on three important NLP tasks. Our experiments demonstrate that on all of the three tasks, we get performance comparable to Exact word pair counts setting and state-of-the-art system. Our method scales to 49 GB of unzipped web data using bounded space of 2 billion counters (8 GB memory).

Sketching of Mirror-Symmetric Shapes

Abstract: This paper presents a system to create mirror-symmetric surfaces from free-form sketches. The system takes as input a hand-drawn sketch and generates a surface whose silhouette approximately matches the input sketch. The input sketch typically consists of a set of curves connected at their endpoints, forming T-junctions and cusps. Our system is able to identify the skewed-mirror and translational symmetry between the hand-drawn curves and uses this information to reconstruct the occluded parts of the surface and its 3D shape.

Single-view sketch based modeling

Abstract: This paper presents a new sketch modeling system that is able to generate complex objects drawn from a unique viewpoint. The user draws the model in an iterative manner, adding simple parts to the existing object until completion. Each part is constructed from two construction lines (lines on the surface of the object that are planar and perpendicular to each other) whose orientation in the 3D space is uniquely determined by the system, and an optional silhouette. The system is then able to produce rough 3D reconstructions of drawings very easily by tracing over a sketch for example. Such models are perfectly suited to investigate their shade or shadow and they can be used as substitutes for more detailed models when the need for quick models is present. The user can also explore shapes directly on the system, refining the shape on the go in a oversketching way. The creation of models is very efficient, as the user models the shapes directly in the correct pose and orientation. Results show that the system is able to create complex objects without ever having to change the viewpoint.