This paper surveys the field of augmented reality AR, in which 3D virtual objects are integrated into a 3D real environment in real time. It describes the medical, manufacturing, visualization, path planning, entertainment, and military applications that have been explored. This paper describes the characteristics of augmented reality systems, including a detailed discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective augmented reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. This survey provides a starting point for anyone interested in researching or using augmented reality.

/pdf/a-survey-of-augmented-reality-17irtaxt7z.pdf

A survey of augmented reality

Paul Milgram received the BASc degree from the University of Toronto in 1970, the MSEE degree from the Technion (Israel) in 1973 and the PhD degree from the University of Toronto in 1980 From 1980 to 1982 he was a ZWO Visiting Scientist and a NATO Postdoctoral in the Netherlands, researching automobile driving behaviour From 1982 to 1984 he was a Senior Research Engineer in Human Engineering at the National Aerospace Laboratory (NLR) in Amsterdam, where his work involved the modelling of aircraft flight crew activity, advanced display concepts and control loops with human operators in space teleoperation Since 1986 he has worked at the Industrial Engineering Department of the University of Toronto, where he is currently an Associate Professor and Coordinator of the Human Factors Engineering group He is also cross appointed to the Department of Psychology In 1993-94 he was an invited researcher at the ATR Communication Systems Research Laboratories, in Kyoto, Japan His research interests include display and control issues in telerobotics and virtual environments, stereoscopic video and computer graphics, cognitive engineering, and human factors issues in medicine He is also President of Translucent Technologies, a company which produces "Plato" liquid crystal visual occlusion spectacles (of which he is the inventor), for visual and psychomotor research

/pdf/a-taxonomy-of-mixed-reality-visual-displays-54v3x9pjl3.pdf

A Taxonomy of Mixed Reality Visual Displays

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

Methods and systems for processing touch inputs are disclosed. The invention in one respect includes reading data from a multipoint sensing device such as a multipoint touch screen where the data pertains to touch input with respect to the multipoint sensing device, and identifying at least one multipoint gesture based on the data from the multipoint sensing device.

Gestures for touch sensitive input devices

Apparatus and methods are disclosed for simultaneously tracking multiple finger (202-204) and palm (206, 207) contacts as hands approach, touch, and slide across a proximity-sensing, compliant, and flexible multi-touch surface (2). The surface consists of compressible cushion (32), dielectric electrode (33), and circuitry layers. A simple proximity transduction circuit is placed under each electrode to maximize the signal-to-noise ratio and to reduce wiring complexity. Scanning and signal offset removal on electrode array produces low-noise proximity images. Segmentation processing of each proximity image constructs a group of electrodes corresponding to each distinguishable contacts and extracts shape, position and surface proximity features for each group. Groups in successive images which correspond to the same hand contact are linked by a persistent path tracker (245) which also detects individual contact touchdown and liftoff. Classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Method and apparatus for integrating manual input

This work explores a new direction in utilizing eye gaze for computer input. Gaze tracking has long been considered as an alternative or potentially superior pointing method for computer input. We believe that many fundamental limitations exist with traditional gaze pointing. In particular, it is unnatural to overload a perceptual channel such as vision with a motor control task. We therefore propose an alternative approach, dubbed MAGIC (Manual And Gaze Input Cascaded) pointing. With such an approach, pointing appears to the user to be a manual task, used for fine manipulation and selection. However, a large portion of the cursor movement is eliminated by warping the cursor to the eye gaze area, which encompasses the target. Two specific MAGIC pointing techniques, one conservative and one liberal, were designed, analyzed, and implemented with an eye tracker we developed. They were then tested in a pilot study. This early- stage exploration showed that the MAGIC pointing techniques might offer many advantages, including reduced physical effort and fatigue as compared to traditional manual pointing, greater accuracy and naturalness than traditional gaze pointing, and possibly faster speed than manual pointing. The pros and cons of the two techniques are discussed in light of both performance data and subjective reports.

/pdf/manual-and-gaze-input-cascaded-magic-pointing-2hny9xx29s.pdf

Manual and gaze input cascaded (MAGIC) pointing

As a developing discipline, research results in the field of human computer interaction (HCI) tends to be "soft". Many workers in the field have argued that the advancement of HCI lies in "hardening" the field with quantitative and robust models. In reality, few theoretical, quantitative tools are available in user interface research and development. A rare exception to this is Fitts' law. Extending information theory to human perceptual-motor system, Paul Fitts (1954) found a logarithmic relationship that models speed accuracy tradeoffs in aimed movements. A great number of studies have verified and / or applied Fitts' law to HCI problems, such as pointing performance on a screen, making Fitts' law one of the most intensively studied topic in the HCI literature.

/pdf/beyond-fitts-law-models-for-trajectory-based-hci-tasks-52zct72gz4.pdf

Beyond Fitts' law: models for trajectory-based HCI tasks

Zhai and Kristensson (2003) presented a method of speed-writing for pen-based computing which utilizes gesturing on a stylus keyboard for familiar words and tapping for others. In SHARK2:, we eliminated the necessity to alternate between the two modes of writing, allowing any word in a large vocabulary (e.g. 10,000-20,000 words) to be entered as a shorthand gesture. This new paradigm supports a gradual and seamless transition from visually guided tracing to recall-based gesturing. Based on the use characteristics and human performance observations, we designed and implemented the architecture, algorithms and interfaces of a high-capacity multi-channel pen-gesture recognition system. The system's key components and performance are also reported.

/pdf/shark2-a-large-vocabulary-shorthand-writing-system-for-pen-2uxe51xqmm.pdf

SHARK2: a large vocabulary shorthand writing system for pen-based computers

Bare hand pointing on touch screens both benefits and suffers from the nature of direct input. This work explores techniques to overcome its limitations. Our goal is to design interaction tools allowing pixel level pointing in a fast and efficient manner. Based on several cycles of iterative design and testing, we propose two techniques: Cross-Keys that uses discrete taps on virtual keys integrated with a crosshair cursor, and an analog Precision-Handle that uses a leverage (gain) effect to amplify movement precision from the user's finger tip to the end cursor. We conducted a formal experiment with these two techniques, in addition to the previously known Zoom-Pointing and Take-Off as baseline anchors. Both subjective and performance measurements indicate that Precision-Handle and Cross-Keys complement existing techniques for touch screen interaction.

High precision touch screen interaction

Today's graphical interactive systems largely depend upon pointing actions, i.e. entering an object and selecting it. In this paper we explore whether an alternate paradigm --- crossing boundaries --- may substitute or complement pointing as another fundamental interaction method. We describe an experiment in which we systematically evaluate two target-pointing tasks and four goal-crossing tasks, which differ by the direction of the movement variability constraint (collinear vs. orthogonal) and by the nature of the action (pointing vs. crossing, discrete vs. continuous). We found that participants' temporal performance in each of the six tasks was dependent on the index of difficulty formulated in the same way as in Fitts' law, but that the parameters differ by task. We also found that goal crossing completion time was shorter or no longer than pointing performance under the same index of difficulty. These regularities, as well as qualitative characterizations of crossing actions and their application in HCI, lay the foundation for designing crossing-based user interfaces

Shumin Zhai

Papers

Manual and gaze input cascaded (MAGIC) pointing

Beyond Fitts' law: models for trajectory-based HCI tasks

SHARK2: a large vocabulary shorthand writing system for pen-based computers

High precision touch screen interaction

More than dotting the i's --- foundations for crossing-based interfaces