A scheme is developed for classifying the types of motion perceived by a humanlike robot. It is assumed that the robot receives visual images of the scene using a perspective system model. Equations, theorems, concepts, clues, etc., relating the objects, their positions, and their motion to their images on the focal plane are presented. >

http://ieeexplore.ieee.org/iel2/815/3148/00100651.pdf

Robot vision

A remote controlled robot system that includes a robot and a remote control station. The robot includes a binaural microphone system that is coupled to a speaker system of the remote control station. The binaural microphone system may include a pair of microphones located at opposite sides of a robot head. the location of the microphones roughly coincides with the location of ears on a human body. Such microphone location creates a mobile robot that more effectively simulates the tele-presence of an operator of the system. The robot may include two different microphone systems and the ability to switch between systems. For example, the robot may also include a zoom camera system and a directional microphone. The directional microphone may be utilized to capture sound from a direction that corresponds to an object zoomed upon by the camera system.

Mobile tele-presence system with a microphone system

A remote controlled robot with a head that supports a monitor and is coupled to a mobile platform. The mobile robot also includes an auxiliary camera coupled to the mobile platform by a boom. The mobile robot is controlled by a remote control station. By way of example, the robot can be remotely moved about an operating room. The auxiliary camera extends from the boom so that it provides a relatively close view of a patient or other item in the room. An assistant in the operating room may move the boom and the camera. The boom may be connected to a robot head that can be remotely moved by the remote control station.

Telepresence robot with a camera boom

A mobile robot that includes a drive system, a controller in communication with the drive system, and a volumetric point cloud imaging device supported above the drive system at a height of greater than about one feet above the ground and directed to be capable of obtaining a point cloud from a volume of space that includes a floor plane in a direction of movement of the mobile robot. The controller receives point cloud signals from the imaging device and issues drive commands to the drive system based at least in part on the received point cloud signals.

Mobile human interface robot

A telepresence robot may include a drive system, a control system, an imaging system, and a mapping module. The mapping module may access a plan view map of an area and tags associated with the area. In various embodiments, each tag may include tag coordinates and tag information, which may include a tag annotation. A tag identification system may identify tags within a predetermined range of the current position and the control system may execute an action based on an identified tag whose tag information comprises a telepresence robot action modifier. The telepresence robot may rotate an upper portion independent from a lower portion. A remote terminal may allow an operator to control the telepresence robot using any combination of control methods, including by selecting a destination in a live video feed, by selecting a destination on a plan view map, or by using a joystick or other peripheral device.

Interfacing with a mobile telepresence robot

A system is presented that creates vision-based, three-dimensional control of a multiple-degree-of-freedom dexterous robot, without special calibration of the vision system, the robot, or any of the constituent parts of the system, and that allows high-level human supervision or direction of the robot. The human operator uses a graphical user interface (GUI) to point and click on an image of the surface of the object with which the robot is to interact. Directed at this surface is the stationary selection camera, which provides the image for the GUI, and at least one other camera. A laser pointer is panned and tilted so as to create, in each participating camera space, targets associated with surface junctures that the user has selected in the selection camera. Camera-space manipulation is used to control the internal degrees of freedom of the robot such that selected points on the robot end member move relative to selected surface points in a way that is consistent with the desired robot operation. As per the requirement of camera-space manipulation, the end member must have features, or “cues”, with known location relative to the controlled end-member points, that can be located in the images or camera spaces of participant cameras. The system is extended to simultaneously control tool orientation relative to the surface normal and/or relative to user-selected directions tangent to the surface. The system is extended in various ways to allow for additional versatility of application.

Means and method of robot control relative to an arbitrary surface using camera-space manipulation

Automatic visual guidance of a forklift engaging a pallet

The present application provides a system for enabling instrument placement from distances on the order of five meters, for example, and increases accuracy of the instrument placement relative to visually-specified targets. The system provides precision control of a mobile base of a rover and onboard manipulators (e.g., robotic arms) relative to a visually-specified target using one or more sets of cameras. The system automatically compensates for wheel slippage and kinematic inaccuracy ensuring accurate placement (on the order of 2 mm, for example) of the instrument relative to the target. The system provides the ability for autonomous instrument placement by controlling both the base of the rover and the onboard manipulator using a single set of cameras. To extend the distance from which the placement can be completed to nearly five meters, target information may be transferred from navigation cameras (used for long-range) to front hazard cameras (used for positioning the manipulator).

/pdf/method-and-system-for-providing-autonomous-control-of-a-2xnrvie4cb.pdf

Method and system for providing autonomous control of a platform

The invention is a method of using computer vision to control systems consisting of a combination of holonomic and nonholonomic degrees of freedom such as a wheeled rover equipped with a robotic arm, a forklift, and earth-moving equipment such as a backhoe or a front-loader. Using vision sensors mounted on the mobile system and the manipulator, the system establishes a relationship between the internal joint configuration of the holonomic degrees of freedom of the manipulator and the appearance of features on the manipulator in the reference frames of the vision sensors. Then, the system, perhaps with the assistance of an operator, identifies the locations of the target object in the reference frames of the vision sensors. Using this target information, along with the relationship described above, the system determines a suitable trajectory for the nonholonomic degrees of freedom of the base to follow towards the target object. The system also determines a suitable pose or series of poses for the holonomic degrees of freedom of the manipulator. With additional visual samples, the system automatically updates the trajectory and final pose of the manipulator so as to allow for greater precision in the overall final position of the system.

/pdf/mobile-camera-space-manipulation-38a75q74ir.pdf

Mobile camera-space manipulation

This paper presents a vision-guided control method called mobile camera-space manipulation (MCSM) that enables a robotic forklift vehicle to engage pallets based on a pallet’s actual current location by using feedback from vision sensors that are part of the robotic forklift. MCSM is capable of high precision mobile manipulation control without relying on strict camera calibration. The paper contains development of the method as well as experimental results with a forklift prototype in actual pallet engagement tasks. The technology could be added to AGV (automatically guided vehicle) systems enabling them to engage arbitrarily located pallets. It also could be added to standard forklifts as an operator assist capability.

Michael J. Seelinger

Papers

Means and method of robot control relative to an arbitrary surface using camera-space manipulation

Automatic visual guidance of a forklift engaging a pallet

Method and system for providing autonomous control of a platform

Mobile camera-space manipulation

Automatic Pallet Engagment by a Vision Guided Forklift