This article provides a tutorial introduction to visual servo control of robotic manipulators. Since the topic spans many disciplines our goal is limited to providing a basic conceptual framework. We begin by reviewing the prerequisite topics from robotics and computer vision, including a brief review of coordinate transformations, velocity representation, and a description of the geometric aspects of the image formation process. We then present a taxonomy of visual servo control systems. The two major classes of systems, position-based and image-based systems, are then discussed in detail. Since any visual servo system must be capable of tracking image features in a sequence of images, we also include an overview of feature-based and correlation-based methods for tracking. We conclude the tutorial with a number of observations on the current directions of the research field of visual servo control.

/pdf/a-tutorial-on-visual-servo-control-5cy6rje63g.pdf

A tutorial on visual servo control

Preface. 1. Introduction. 2. Rigid Motions and Homogeneous Transformations. 3. Forward and Inverse Kinematics. 4. Velocity Kinematics-The Jacobian. 5. Path and Trajectory Planning. 6. Independent Joint Control. 7. Dynamics. 8. Multivariable Control. 9. Force Control. 10. Geometric Nonlinear Control. 11. Computer Vision. 12. Vision-Based Control. Appendix A: Trigonometry. Appendix B: Linear Algebra. Appendix C: Dynamical Systems. Appendix D: Lyapunov Stability. Index.

/pdf/robot-modeling-and-control-2n3memrp6g.pdf

Robot Modeling and Control

The authors present algorithms for robotic (eye-in-hand configuration) real-time visual tracking of arbitrary 3D objects traveling at unknown velocities in a 2D space (depth is given as known). Visual tracking is formulated as a problem of combining control with computer vision. A mathematical formulation of the control problem that includes information from a novel feedback vision sensor and represents everything with respect to the camera frame is presented. The sum-of-squared differences (SSD) optical flow is used to compute the vector of discrete displacements each instant of time. These displacements can be fed either directly to a PI (proportional-integral) controller or to a pole assignment controller or discrete steady-state Kalman filter. In the latter case, the Kalman filter calculates the estimated values of the system's states and the exogenous disturbances, and a discrete LQG (linear-quadratic Gaussian) controller computes the desired motion of the robotic system. The outputs of the controllers are sent to the Cartesian robotic controller. Performance results are presented. >

Visual tracking of a moving target by a camera mounted on a robot: a combination of control and vision

The mathematics of coordinated control of prosthetic arms and manipulators.

This paper attempts to present a comprehensive summary of research results in the use of visual information to control robot manipulators and related mechanisms. An extensive bibliography is provided which also includes important papers from the elemental disciplines upon which visual servoing is based. The research results are discussed in terms of historical context, common-ality of function, algorithmic approach and method of implementation.

Visual control of robot manipulators – a review

The authors develop methodologies for the automatic selection of image features to be used to visually control the relative position and orientation (pose) between the end-effector of an eye-in-hand robot and a workpiece. A resolved motion rate control scheme is used to update the robot's pose based on the position of three features in the camera's image. The selection of these three features depends on a blend of image recognition and control criteria. The image recognition criteria include feature robustness, completeness, cost of feature extraction, and feature uniqueness. The control criteria include system observability, controllability, and sensitivity. A weighted criteria function is used to select the combination of image features that provides the best control of the end-effector of a general six-degrees-of-freedom manipulator. Both computer simulations and laboratory experiments on a PUMA robot arm were conducted to verify the performance of the feature-selection criteria. >

O.R. Mitchell

Papers

Weighted selection of image features for resolved rate visual feedback control