This paper surveys the field of augmented reality AR, in which 3D virtual objects are integrated into a 3D real environment in real time. It describes the medical, manufacturing, visualization, path planning, entertainment, and military applications that have been explored. This paper describes the characteristics of augmented reality systems, including a detailed discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective augmented reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. This survey provides a starting point for anyone interested in researching or using augmented reality.

/pdf/a-survey-of-augmented-reality-17irtaxt7z.pdf

A survey of augmented reality

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

http://library.utia.cas.cz/prace/20030125.pdf

Image registration methods: a survey

Recent studies of eye movements in reading and other information processing tasks, such as music reading, typing, visual search, and scene perception, are reviewed. The major emphasis of the review is on reading as a specific example of cognitive processing. Basic topics discussed with respect to reading are (a) the characteristics of eye movements, (b) the perceptual span, (c) integration of information across saccades, (d) eye movement control, and (e) individual differences (including dyslexia). Similar topics are discussed with respect to the other tasks examined. The basic theme of the review is that eye movement data reflect moment-to-moment cognitive processes in the various tasks examined. Theoretical and practical considerations concerning the use of eye movement data are also discussed.

/pdf/eye-movements-in-reading-and-information-processing-20-years-3z5ay8hdwc.pdf

Eye movements in reading and information processing: 20 years of research.

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

This paper describes targeted reaching experiments conducted using a new augmented reality system. Combining a large-workspace immersive virtual environment with physical force feedback, the system distorted subjects' movements using a viscous curl force field. Following previous experiments using a different robot, half the subjects were constrained to horizontal, planar movements. The remaining subjects performed unconstrained movements throughout the 3D workspace. Examining after-effects as an indication of learning, we found that constrained subjects learned the force field. However, it was difficult to detect whether the unconstrained subjects learned forces of identical magnitude. Our results found that force fields strengths eliciting constrained 2D adaptation have difficulty exhibiting after-effects for unconstrained 3D movements. The increased motor variability for 3D reaching movements requires consideration for future experimental design.

/pdf/comparing-adaptation-of-constrained-and-unconstrained-4pcnrw8eio.pdf

Comparing adaptation of constrained and unconstrained movements in three dimensions

Smooth pursuit eye movements were studied in three patients with alcoholic Korsakoff's syndrome, one with Wernicke's encephalopathy, and an age-matched control. Horizontal smooth pursuit eye movements were abnormal in all patients: peak eye velocity and the ability to sustain smooth eye velocity were reduced. Also, smooth pursuit gain began to decrease at relatively low target velocities (i.e., 8–10°). These data demonstrate a severe disturbance in smooth pursuit function long after the clinically apparent oculomotor abnormalities have passed.

Oculomotor function in Wernicke-Korsakoff's syndrome: II. Smooth pursuit eye movements.

Propagation of signals across the cerebral cortex is a core component of many cognitive processes and is generally thought to be mediated by direct intracortical connectivity. The thalamus, by contrast, is considered to be devoid of internal connections and organized as a collection of parallel inputs to the cortex. Here, we provide evidence that “open-loop” intrathalamic connections involving the thalamic reticular nucleus (TRN) can support propagation of oscillatory activity across the cortex. Recent studies support the existence of open-loop thalamo-reticulo-thalamic (TC-TRN-TC) synaptic motifs in addition to traditional closed-loop architectures. We hypothesized that open-loop structural modules, when connected in series, might underlie thalamic and, therefore cortical, signal propagation. Using a supercomputing platform to simulate thousands of permutations of a thalamo-reticular-cortical network and allowing select synapses to vary both by class and individually, we evaluated the relative capacities of closed- and open-loop TC-TRN-TC synaptic configurations to support both propagation and oscillation. We observed that 1) signal propagation was best supported in networks possessing strong open-loop TC-TRN-TC connectivity; 2) intrareticular synapses were neither primary substrates of propagation nor oscillation; and 3) heterogeneous synaptic networks supported more robust propagation of oscillation than their homogeneous counterparts. These findings suggest that open-loop heterogeneous intrathalamic architectures complement direct intracortical connectivity to facilitate cortical signal propagation. Significance Statement Interactions between the dorsal thalamus and thalamic reticular nucleus (TRN) are speculated to contribute to phenomena such as arousal, attention, sleep, and seizures. Despite the importance of the TRN, the synaptic microarchitectures forming the basis for dorsal thalamus-TRN interactions are not fully understood. The computational neural model we present incorporates “open-loop” thalamo-reticular-thalamic (TC-TRN-TC) synaptic motifs, which have been experimentally observed. We elucidate how open-loop motifs possess the capacity to shape the propagative properties of signals intrinsic to the thalamus and evaluate the wave dynamics they support relative to closed-loop TC-TRN-TC pathways and intrareticular synaptic connections. Our model also generates predictions regarding how different spatial distributions of reticulothalamic and intrareticular synapses affect these signaling properties.

/pdf/propagation-of-cortical-activity-via-open-loop-intrathalamic-2l3lxjtcsi.pdf

Propagation of cortical activity via open-loop intrathalamic architectures: a computational analysis

We present a coordinated ensemble of scalable computing techniques to accelerate a number of key tasks needed for vision-based gesture interaction, by using the cluster driving a large display system. A hybrid strategy that partitions the scanning task of a frame image by both region and scale is proposed. Based on this hybrid strategy, a novel data structure called a scanning tree is designed to organize the computing nodes. The level of effectiveness of the proposed solution was tested by incorporating it into a gesture interface controlling a ultra-high-resolution tiled display wall.

/pdf/scalable-vision-based-gesture-interaction-for-cluster-driven-1rsecagcmi.pdf

Scalable Vision-based Gesture Interaction for Cluster-driven High Resolution Display Systems

BACKGROUND Stroke recovery studies have shown the efficacy of bimanual training on upper limb functional recovery and others have shown the efficacy of feedback technology that augments error. OBJECTIVE In a double-blinded randomized controlled study (N = 26), we evaluated the short-term effects of bilateral arm training to foster functional recovery of a hemiparetic arm, with half of our subjects unknowingly also receiving error augmentation (where errors were visually and haptically enhanced by a robot). METHODS Twenty-six individuals with chronic stroke were randomly assigned to practice an equivalent amount of bimanual reaching either with or without error augmentation. Participants were instructed to coordinate both arms while reaching to two targets (one for each arm) in three 45-minute treatments per week for two weeks, with a follow-up visit after one week without treatment. RESULTS Subjects' 2-week gains in Fugl-Meyer score averaged 2.92, and we also observed improvements Wolf Motor Functional Ability Scale average 0.21, and Motor Activity Log of 0.58 for quantity and 0.63 for quality of life scores. The extra benefit of error augmentation over the three weeks became apparent in Fugl-Meyer score only after removing an outlier from consideration. CONCLUSIONS This modest advantage of error augmentation was detectable over a short interval encouraging further research in interactive self-rehabilitation systems that can enhance error motor recovery.

Robert V. Kenyon

Papers

Comparing adaptation of constrained and unconstrained movements in three dimensions

Oculomotor function in Wernicke-Korsakoff's syndrome: II. Smooth pursuit eye movements.

Propagation of cortical activity via open-loop intrathalamic architectures: a computational analysis

Scalable Vision-based Gesture Interaction for Cluster-driven High Resolution Display Systems

Error-augmented bimanual therapy for stroke survivors.