This paper surveys the field of augmented reality AR, in which 3D virtual objects are integrated into a 3D real environment in real time. It describes the medical, manufacturing, visualization, path planning, entertainment, and military applications that have been explored. This paper describes the characteristics of augmented reality systems, including a detailed discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective augmented reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. This survey provides a starting point for anyone interested in researching or using augmented reality.

/pdf/a-survey-of-augmented-reality-17irtaxt7z.pdf

A survey of augmented reality

http://ee.sharif.edu/~miap/Files/A%20Survey%20of%20Medical%20Image%20Registration.pdf

A survey of medical image registration.

Many applications require tracking of complex 3D objects. These include visual servoing of robotic arms on specific target objects, Augmented Reality systems that require real-time registration of the object to be augmented, and head tracking systems that sophisticated interfaces can use. Computer Vision offers solutions that are cheap, practical and non-invasive.This survey reviews the different techniques and approaches that have been developed by industry and research. First, important mathematical tools are introduced: Camera representation, robust estimation and uncertainty estimation. Then a comprehensive study is given of the numerous approaches developed by the Augmented Reality and Robotics communities, beginning with those that are based on point or planar fiducial marks and moving on to those that avoid the need to engineer the environment by relying on natural features such as edges, texture or interest. Recent advances that avoid manual initialization and failures due to fast motion are also presented. The survery concludes with the different possible choices that should be made when implementing a 3D tracking system and a discussion of the future of vision-based 3D tracking.Because it encompasses many computer vision techniques from low-level vision to 3D geometry and includes a comprehensive study of the massive literature on the subject, this survey should be the handbook of the student, the researcher, or the engineer who wants to implement a 3D tracking system.

/pdf/monocular-model-based-3d-tracking-of-rigid-objects-a-survey-1tb409lav4.pdf

Monocular Model-Based 3D Tracking of Rigid Objects: A Survey

The detection and recognition of objects in images is a key research topic in the computer vision community Within this area, face recognition and interpretation has attracted increasing attention owing to the possibility of unveiling human perception mechanisms, and for the development of practical biometric systems This book and the accompanying website, focus on template matching, a subset of object recognition techniques of wide applicability, which has proved to be particularly effective for face recognition applications Using examples from face processing tasks throughout the book to illustrate more general object recognition approaches, Roberto Brunelli: examines the basics of digital image formation, highlighting points critical to the task of template matching; presents basic and advanced template matching techniques, targeting grey-level images, shapes and point sets; discusses recent pattern classification paradigms from a template matching perspective; illustrates the development of a real face recognition system; explores the use of advanced computer graphics techniques in the development of computer vision algorithms Template Matching Techniques in Computer Vision is primarily aimed at practitioners working on the development of systems for effective object recognition such as biometrics, robot navigation, multimedia retrieval and landmark detection It is also of interest to graduate students undertaking studies in these areas

Template Matching Techniques in Computer Vision: Theory and Practice

Wearable computing moves computation from the desktop to the user. We are forming a community of networked, wearable-computer users to explore, over a long period, the augmented realities that these systems can provide. By adapting its behavior to the user's changing environment, a body-worn computer can assist the user more intelligently, consistently, and continuously than a desktop system. A text-based augmented reality, the Remembrance Agent, is presented to illustrate this approach. Video cameras are used both to warp the visual input mediated reality and to sense the user's world for graphical overlay. With a camera, the computer could track the user's finger to act as the system's mouse; perform face recognition; and detect passive objects to overlay 2.5D and 3D graphics onto the real world. Additional apparatus such as audio systems, infrared beacons for sensing location, and biosensors for learning about the wearer's affect are described. With the use of input from these interface devices and sensors, a long-term goal of this project is to model the user's actions, anticipate his or her needs, and perform a seamless interaction between the virtual and physical environments.

Augmented reality through wearable computing

This paper presents computer vision based techniques for object registration, real-time tracking, and image overlay. The capability can be used to superimpose registered images such as those from CT or MRI onto a video image of a patient’s body. Real-time object registration enables an image to be overlaid consistently onto objects even while the object or the viewer is moving. The video image of a patient’s body is used as input for object registration. Reliable real-time object registration at frame rate (30 Hz) is realized by a combination of techniques, including template matching based feature detection, feature correspondence by geometric constraints, and pose calculation of objects from feature positions in the image. Two types of image overlay systems are presented. The first one registers objects in the image and projects preoperative model data onto a raw camera image. The other computes the position of image overlay directly from 2D feature positions without any prior models. With the techniques developed in this paper, interactive video, which transmits images of a patient to the expert and sends them back with some image overlay, can be realized.

Vision-Based Object Registration for Real-Time Image Overlay

We present a fast pattern matching algorithm with a large set of templates. The algorithm is based on the typical template matching speeded up by the dual decomposition; the Fourier transform and the Karhunen-Loeve transform. The proposed algorithm is appropriate for the search of an object with unknown distortion within a short period. Patterns with different distortion differ slightly from each other and are highly correlated. The image vector subspace required for effective representation can be defined by a small number of eigenvectors derived by the Karhunen-Loeve transform. A vector subspace spanned by the eigenvectors is generated, and any image vector in the subspace is considered as a pattern to be recognized. The pattern matching of objects with unknown distortion is formulated as the process to extract the portion of the input image, find the pattern most similar to the extracted portion in the subspace, compute normalized correlation between them at each location in the input image, and find the location with the best score. Searching for objects with unknown distortion requires vast computation. The formulation above makes it possible to decompose highly correlated reference images into eigenvectors, as well as to decompose images in frequency domain, and to speed up the process significantly.

/pdf/use-of-fourier-and-karhunen-loeve-decomposition-for-fast-4l9tgcz5q3.pdf

Use of Fourier and Karhunen-Loeve decomposition for fast pattern matching with a large set of templates

We present the concept that for uniformly rotated images, the optimal approximation of the images can be obtained by computing the basis vectors for the discrete cosine transform (DCT) of the original image in polar coordinates, and that the images can be represented as linear combinations of the basis vectors.

/pdf/optimal-approximation-of-uniformly-rotated-images-2likglls4q.pdf

Optimal approximation of uniformly rotated images: relationship between Karhunen-Loeve expansion and discrete cosine transform

A new master-slave manipulator (MSM) with different configurations and different degrees of freedom (DOF) has been developed. It consists of a general master arm using pantograph mechanisms and a seven degree of freedom slave arm with a redundant joint. Symmetric bilateral control using a virtual internal model and redundant arm control with a singular configuration are used to improve the maneuverability of the MSM. The developed MSM has been confirmed to have good maneuverability from basic experiments. This paper describes the mechanisms, control methods, and experimental results of the MSM.

Development of an advanced master-slave manipulator using a pantograph master arm and a redundant slave arm

A new approach to the static analysis of the link compliances is presented for appropriate compliance control of a flexible manipulator. Unlike a rigid manipulator, there exists the interaction between the servo flexibility and link flexibility in a flexible manipulator system. Therefore, it is important for smooth control to understand properties of the link compliances and make good conditions of the system compliances. For this purpose, the authors derived the generalized compliance matrix, compliance ellipsoid and compliance center of the link flexibility. It was found that there was an adequate attitude of a manipulator for excluding the influence of the link compliance and establishing tasks smoothly. The validity of this analytic method was investigated by experiments of the round peg-hole assembly. A two-link flexible SCARA manipulator system was constructed for this purpose. This manipulator was 1.5 m long. The results showed that the smooth and precise assembly was established by selecting adequate attitude and setting servo compliance conditions more flexible than the link compliances.

Michihiro Uenohara

Papers

Vision-Based Object Registration for Real-Time Image Overlay

Use of Fourier and Karhunen-Loeve decomposition for fast pattern matching with a large set of templates

Optimal approximation of uniformly rotated images: relationship between Karhunen-Loeve expansion and discrete cosine transform

Development of an advanced master-slave manipulator using a pantograph master arm and a redundant slave arm

Compliance control for a two-link flexible manipulator.