This paper surveys the field of augmented reality AR, in which 3D virtual objects are integrated into a 3D real environment in real time. It describes the medical, manufacturing, visualization, path planning, entertainment, and military applications that have been explored. This paper describes the characteristics of augmented reality systems, including a detailed discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective augmented reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. This survey provides a starting point for anyone interested in researching or using augmented reality.

/pdf/a-survey-of-augmented-reality-17irtaxt7z.pdf

A survey of augmented reality

Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

This article provides a tutorial introduction to visual servo control of robotic manipulators. Since the topic spans many disciplines our goal is limited to providing a basic conceptual framework. We begin by reviewing the prerequisite topics from robotics and computer vision, including a brief review of coordinate transformations, velocity representation, and a description of the geometric aspects of the image formation process. We then present a taxonomy of visual servo control systems. The two major classes of systems, position-based and image-based systems, are then discussed in detail. Since any visual servo system must be capable of tracking image features in a sequence of images, we also include an overview of feature-based and correlation-based methods for tracking. We conclude the tutorial with a number of observations on the current directions of the research field of visual servo control.

/pdf/a-tutorial-on-visual-servo-control-5cy6rje63g.pdf

A tutorial on visual servo control

Preface. 1. Introduction. 2. Rigid Motions and Homogeneous Transformations. 3. Forward and Inverse Kinematics. 4. Velocity Kinematics-The Jacobian. 5. Path and Trajectory Planning. 6. Independent Joint Control. 7. Dynamics. 8. Multivariable Control. 9. Force Control. 10. Geometric Nonlinear Control. 11. Computer Vision. 12. Vision-Based Control. Appendix A: Trigonometry. Appendix B: Linear Algebra. Appendix C: Dynamical Systems. Appendix D: Lyapunov Stability. Index.

/pdf/robot-modeling-and-control-2n3memrp6g.pdf

Robot Modeling and Control

https://www.cs.cmu.edu/~illah/PAPERS/socialroboticssurvey.pdf

A survey of socially interactive robots

A purposeful change of camera parameters or “active vision” can be used to improve the process of extracting visual information. Thus if a robot visual servo loop incorporates active vision, it can lead to a better performance while increasing the scope of the control tasks. Although significant advances have been made in this direction, much of the potential improvement is still unrealized. This chapter discusses the advantages of using active vision for visual servoing. It reviews some of the past research in active vision relevant to visual servoing, with the aim of improving: (1) the measurement of image parameters, (2) the process of interpreting the image parameters in terms of the corresponding world parameters, and (3) the control of a robot in terms of the visual information extracted.

Role of active vision in optimizing visual feedback for robot control

Editorial: introduction to the special issue on human modeling, analysis, and synthesis

We present a multiple object tracking framework that employs two common methods for tracking and image matching, namely Multiple Hypothesis Tracking (MHT) and Hausdorff image matching. We use the MHT algorithm to track image edges simultaneously. This algorithm is capable of tracking multiple edges with limited occlusions and is suitable for resolving any data association uncertainty caused by background clutter and closely-spaced edges. We use the Hausdorff matching algorithm to organize individual edges into objects given their two-dimensional models. The combined technique provides a robust probabilistic tracking framework which is capable of tracking complex objects in cluttered background in video sequences.

Multiple complex object tracking using a combined technique

Active vision refers to a purposeful change in the camera setup to aid the processing of visual information. An important issue in using active vision is the need to represent the 3D environment in a manner that is invariant to changing camera configurations. Conventional methods require precise knowledge of various camera parameters in order to build this representation. However, these parameters are prone to calibration errors. This motivates us to explore a neural network based approach using Vector Associative Map to learn the invariant representation of 3D point targets for active vision. An efficient learning scheme is developed that is suitable for robotic implementation. The representation thus learned is also independent of the intrinsic parameters of the imaging system, making it immune to systematic calibration errors. To evaluate the effectiveness of this scheme, computer simulations were first performed using a detailed model of the University of Illinois Active Vision System (UIAVS). This is followed by an experimental verification on the actual UIAVS. Several robotic applications are then explored that utilize the invariance property of the learned representation. These applications include motion detection, active vision based robot control, robot motion planning, and saccade sequence planning.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Robotic applications of VAM-based invariant representation for active vision

The perceptual control manifold is a concept that extends the notion of the robot configuration space to include sensor feedback for robot motion planning. In this paper, we propose a framework for sensor-based robot motion planning using the topology representing network algorithm to develop a learned representation of the perceptual control manifold. The topology preserving features of the neural network lend themselves to yield, after learning, a diffusion-based path planning strategy for flexible obstacle avoidance. Simulations on path control and flexible obstacle avoidance demonstrate the feasibility of this approach for motion planning and illustrate the potential for further robotic applications.

Rajeev Sharma

Papers

Role of active vision in optimizing visual feedback for robot control

Editorial: introduction to the special issue on human modeling, analysis, and synthesis

Multiple complex object tracking using a combined technique

Robotic applications of VAM-based invariant representation for active vision

Learning the perceptual control manifold for sensor-based robot path planning