Humans have used sketching to depict our visual world since prehistoric times. Even today, sketching is possibly the only rendering technique readily available to all humans. This paper is the first large scale exploration of human sketches. We analyze the distribution of non-expert sketches of everyday objects such as 'teapot' or 'car'. We ask humans to sketch objects of a given category and gather 20,000 unique sketches evenly distributed over 250 object categories. With this dataset we perform a perceptual study and find that humans can correctly identify the object category of a sketch 73% of the time. We compare human performance against computational recognition methods. We develop a bag-of-features sketch representation and use multi-class support vector machines, trained on our sketch dataset, to classify sketches. The resulting recognition method is able to identify unknown sketches with 56% accuracy (chance is 0.4%). Based on the computational model, we demonstrate an interactive sketch recognition system. We release the complete crowd-sourced dataset of sketches to the community.

/pdf/how-do-humans-sketch-objects-29vm54gqwr.pdf

How do humans sketch objects

One of the common problems associated with large computer-based information systems is the relatively small window through which an information space can be viewed. Increasing interest in recent years has been focused on the development of distortion-oriented presentation techniques to address this problem. However, the growing number of new terminologies and techniques developed have caused considerable confusion to the graphical user interface designer, consequently making the comparison of these presentation techniques and generalization of empirical results of experiments with them very difficult, if not impossible. This article provides a taxonomy of distortion-oriented techniques which demonstrates clearly their underlying relationships. A unified theory is presented to reveal their roots and origins. Issues relating to the implementation and performance of these techniques are also discussed.

/pdf/a-review-and-taxonomy-of-distortion-oriented-presentation-3728ewc2jd.pdf

A review and taxonomy of distortion-oriented presentation techniques

Apparatus is disclosed for generating control signals for the manipulation of virtual objects in a computer system according to the gestures and positions of an operator's hand or other body part. The apparatus includes a glove worn on the hand which includes sensors for detecting the gestures of the hand, as well as hand position sensing means coupled to the glove and to the computer system for detecting the position of the hand with respect to the system. The computer system includes circuitry connected to receive the gesture signals and the hand position signals for generating control signals in response thereto. Typically, the control signals are used to manipulate a graphical representation of the operator's hand which is displayed on a monitor coupled to the computer system, and the graphical representations of the operator's hand manipulates virtual objects or tools also displayed by the computer.

Computer data entry and manipulation apparatus and method

An organization of interaction techniques, based on the user tasks for which the techniques are used, is proposed. It is suggested that task requirements limit the set of techniques that can be considered for a particular application. The characteristics of a variety of techniques are enumerated and many of the considerations important to their effectiveness are discussed. Relevant experimental and experiential comparisons are tabulated.

The human factors of computer graphics interaction techniques

A computer user interface that represents information stored in a computer as reprentation objects located on at least one reference surface is disclosed. A user can magnify or reduce the displayed information by "zooming" in or out. The magnification operation displays a smaller portion of a particular surface, but each object that remains in view is displayed in greater size and/or detail. Similarly, the reduction operation displays a larger portion of a given surface, but each object within such surface is displayed in a smaller size and/or less detail. In order to view one part of the surface from another part of the surface or from another surface, a special type of an object, called a representation portal object, is provided. A graphical description of each object is physically stored in computer memory essentially as a sequence of compressed image representations. The first image in the sequence is a bit-mapped representation of an object as it was created, and each subsequent image is a reduced copy of the first image.

Fractal computer user centerface with zooming capability

The finger as a graphical stylus enjoys a coefficient of friction with glass sufficient to provide input of direction and torque as well as position from a single point. This report describes a pressure-sensitive digitizer (PSD) capable of accepting these force inputs, and discusses a set of five simple input applications used to assess the capabilities of this device. These applications include techniques for specifying vectors, and pushing, pulling, dispersing and reorienting objects with a single touch. Experience gained from these applications demonstrates that touch and pressure sensing open a rich channel for immediate and multi-dimensional interaction.

One-point touch input of vector information for computer displays

Spatial Data Management is a technique for organizing and retrieving information by positioning it in a spatial framework. Data is accessed in a Spatial Data Management System (SDMS) via pictorial representations which are arranged in space and viewed through a computer graphics system. These pictures can be created by an interactive graphical editor, allowing an SDMS to serve as a personal repository of diagrams, text, and photographs. Pictograms can also be generated from data in a symbolic database management system, allowing SDMS to be used as an interface to large, shared databases.A prototype SDMS has been constructed which employs a set of color, raster scan displays driven by a large minicomputer. The user can create and examine data surfaces which are larger than the display screen, traversing a surface and zooming in and out to control the level of detail displayed. The prototype system provides a uniform mechanism for accessing a wide variety of data types in a manner which does not require the use of a formal command or query language.

A prototype Spatial Data Management System

A family of programs has been developed to allow graphical input through continuous digitizing. Drawing data, sampled at a high and constant rate, is compressed and mapped into lines and splines, in two and three dimensions. This is achieved by inferring a particular user's intentions from measures of speed and pressure.Recent experiments have shown that even the most basic inference making cannot rely solely upon knowledge of the user's drawing style, but needs additional knowledge of the subject being drawn, the protocols of its domain, and the stage of development of the user's design. This requirement implies a higher level of machine intelligence than currently exists. An alternate approach is to increase the user's involvement in the recognition process.Contrary to previous efforts to move from sketch to mechanical drawing without human intervention, this paper reports on an interactive system for graphical input in which the user overtly partakes in training the machine and massaging the data at all levels of interpretation. The initial routines for data compression employ parallel functions for extracting such features as bentness, straightness, and endness. These are planned for implementation in microprocessors.Results offer a system for rapid (and enjoyable) graphical input with real-time interpretation, the beginnings of an intelligent tablet.

/pdf/graphical-input-through-machine-recognition-of-sketches-1haz8w1633.pdf

Graphical input through machine recognition of sketches

This paper reports on the design and implementation of a program visualization (PV) environment, intended to offer the user an integrated graphics programming support system. The PV environment will capitalize on recent progress in the graphical representation of information, to provide designers and programmers with both static and dynamic (animated) views of systems. PV is currently being implemented to support programming in C, although large portions of the system are independent of the software development language. In this paper we provide an overview of the PV environment, along with a detailed discussion of the technique used to instrument programs.

Program Visualization: Graphics Support for Software Development

This paper presents an approach to improving the interface between the user and the computer by replacing the typical action-response dialogue with a looser scheme in which inferences are drawn from user actions. The recognition of hand-drawn sketches is offered as a paradigm for use of this technique in the field of computer-aided design. In design problems, the computer offers the capabilities of exteriorization and transformation. Although transformation processes are widely used in design aids, the computer has found little use as a medium of exteriorization because of difficulty of graphical input. The program described here employs a model of the user as a means of inferring user intent as a function of sequence, speed, and pressure. It is proposed that this modeling technique can be applied to interactive graphics systems, leading to a better interface for the solution of design problems.

Christopher F. Herot

Papers

One-point touch input of vector information for computer displays

A prototype Spatial Data Management System

Graphical input through machine recognition of sketches

Program Visualization: Graphics Support for Software Development

Sketch recognition for computer-aided design