A robot system for use in surgical procedures has two movable arms each carried on a wheeled base with each arm having a six of degrees of freedom of movement and an end effector which can be rolled about its axis and an actuator which can slide along the axis for operating different tools adapted to be supported by the effector. Each end effector including optical force sensors for detecting forces applied to the tool by engagement with the part of the patient. A microscope is located at a position for viewing the part of the patient. The position of the tool tip can be digitized relative to fiducial markers visible in an MRI experiment. The workstation and control system has a pair of hand-controllers simultaneously manipulated by an operator to control movement of a respective one or both of the arms. The image from the microscope is displayed on a monitor in 2D and stereoscopically on a microscope viewer. A second MRI display shows an image of the part of the patient the real-time location of the tool. The robot is MRI compatible and can be configured to operate within a closed magnet bore. The arms are driven about vertical and horizontal axes by piezoelectric motors.

Microsurgical robot system

In one embodiment of the invention, a method for a robotic system is disclosed to track one or more robotic instruments. The method includes generating kinematics information for the robotic instrument within a field of view of a camera; capturing image information in the field of view of the camera; and adaptively fusing the kinematics information and the image information together to determine pose information of the robotic instrument. Additionally disclosed is a robotic medical system with a tool tracking sub-system. The tool tracking sub-system receives raw kinematics information and video image information of the robotic instrument to generate corrected kinematics information for the robotic instrument by adaptively fusing the raw kinematics information and the video image information together.

Methods and systems for robotic instrument tool tracking with adaptive fusion of kinematics information and image information

A number of improvements are provided relating to computer aided surgery utilizing an on tool tracking system. The various improvements relate generally to both the methods used during computer aided surgery and the devices used during such procedures. Other improvements relate to the structure of the tools used during a procedure and how the tools can be controlled using the OTT device. Still other improvements relate to methods of providing feedback during a procedure to improve either the efficiency or quality, or both, for a procedure including the rate of and type of data processed depending upon a CAS mode.

On-board tool tracking system and methods of computer assisted surgery

Optical indicia reading apparatus can comprise a microprocessor, a memory, and a plurality of wafer level camera (WLC) modules. Each WLC module can comprise an image sensor and a focusing lens, and optionally, an optical band-pass filter (BPF) configured to pass light waves within a pre-defined wavelength range. Each WLC module can be configured to output a signal representative of the light incident on the image sensor. The optical axis of a first WLC module can be spatially disposed relatively to an optical axis of a second WLC module so that the fields of view (FOVs) of the WLC modules at least partially overlap. The depth of field (DOF) of a first WLC module can at least partially overlap with the DOF of a second WLC module. The optical indicia reading apparatus can be configured to generate an image frame by processing two or more signals of the plurality of signals outputted by the WLC modules. The optical indicia reading apparatus can be further configured to output an image frame comprising an image of decodable indicia and/or decoded message corresponding to the decodable indicia.

Optical indicia reading apparatus with multiple image sensors

Methods and system perform tool tracking during minimally invasive robotic surgery. Tool states are determined using triangulation techniques or a Bayesian filter from either or both non-endoscopically derived and endoscopically derived tool state information, or from either or both non-visually derived and visually derived tool state information. The non- endoscopically derived tool state information is derived from sensor data provided either by sensors associated with a mechanism for manipulating the tool, or sensors capable of detecting identifiable signals emanating or reflecting from the tool and indicative of its position, or external cameras viewing an end of the tool extending out of the body. The endoscopically derived tool state information is derived from image data provided by an endoscope inserted in the body so as to view the tool.

Methods and System for Performing 3-D Tool Tracking by Fusion of Sensor and/or Camera Derived Data During Minimally Invasive Robotic Surgery

A method of three-dimensional handling of an object by a robot using a tool and one camera mounted on the robot is disclosed in which at least six target features which are normal features of the object are selected on the object.

Method and apparatus for single camera 3d vision guided robotics

A system and method for identifying objects using a machine-vision based system are disclosed. Briefly described, one embodiment is a method that captures a first image of at least one object with an image capture device, processes the first captured image to find an object region based on a reference two-dimensional model and determines a three-dimensional pose estimation based on a reference three-dimensional model that corresponds to the reference two-dimensional model and a runtime three-dimensional representation of the object region where a point-to-point relationship between the reference three-dimensional models of the object and the runtime three-dimensional representation of the object region is not necessarily previously known. Thus, two-dimensional information or data is used to segment an image and three-dimensional information or data used to perform three-dimensional pose estimation on a segment of the image.

System and method of three-dimensional pose estimation

A system and method facilitate machine-vision, for example three-dimensional pose estimation for target objects, using one or more images sensors to acquire images of the target object at one or more positions, and to identify features of the target object in the resulting images. A set of equations is set up using sparse model information such as the constancy of distances, angles and enclosed volumes between features. The set of equations may be solved to estimate a 3D pose. The number of positions may be determined based on the number of image sensors, number of features identified, and/or number of known physical relationships between less than all features. A knowledge of physical relationships between various image sensors and/or between features and image sensors may be employed. The pose may be used to transform a robot path, to align the path with a current position of the target object.

Method and apparatus for machine-vision

Briefly described, one embodiment determines pose of an object of interest at a run time by capturing a first image of a first structured light pattern projected onto a first local surface of the object of interest; determining a first run-time data set from the captured first image, wherein the first run-time data set corresponds to information determined from the first structured light pattern projected onto the first local surface; comparing the determined first run-time data set and a corresponding first reference data set, the first reference data set corresponding to an ideal pose of the first local surface on an ideally posed reference object; and determining at least one first degree of constraint that defines a first partial pose of the first local surface, the at least one first degree of constraint based upon the comparison of the first run-time data set with the corresponding first reference data set.

System and method of determining object pose

L'invention concerne un procede de manipulation tridimensionnelle d'un objet au moyen d'un robot, ce procede faisant appel a un outil et a une camera montes sur le robot. Selon ce procede, au moins six caracteristiques cibles constituant des caracteristiques normales de l'objet sont selectionnees sur ledit objet.

Simona Pescaru

Papers

Method and apparatus for single camera 3d vision guided robotics

System and method of three-dimensional pose estimation

Method and apparatus for machine-vision

System and method of determining object pose

Procede et appareil pour robotique guidee par vision 3d au moyen d'une camera unique