An apparatus and method for controlling a robot may scale a motion of a surgical robot based on a type of object gripped by the surgical robot. In the robot controlling method, by scaling the motion of the surgical robot based on the type of object gripped by the surgical robot, the surgical robot may automatically perform the motion on objects using an optimized force although a user does not control a force minutely based on the type of object gripped by the surgical robot.

Apparatus and method for controlling force to be used for motion of surgical robot

A medical robot system, including a robot coupled to an effectuator element with the robot configured for controlled movement and positioning. The system may include a transmitter configured to emit one or more signals, and the transmitter is coupled to an instrument coupled to the effectuator element. The system may further include a motor assembly coupled to the robot and a plurality of receivers configured to receive the one or more signals emitted by the transmitter. A control unit is coupled to the motor assembly and the plurality of receivers, and the control unit is configured to supply one or more instruction signals to the motor assembly. The instruction signals can be configured to cause the motor assembly to selectively move the effectuator element and is further configured to (i) calculate a position of the at least one transmitter by analysis of the signals received by the plurality of receivers; (ii) display the position of the at least one transmitter with respect to the body of the patient; and (iii) selectively control actuation of the motor assembly in response to the signals received by the plurality of receivers.

Surgical robot platform

Active constraints, also known as virtual fixtures, are high-level control algorithms which can be used to assist a human in man-machine collaborative manipulation tasks. The active constraint controller monitors the robotic manipulator with respect to the environment and task, and anisotropically regulates the motion to provide assistance. The type of assistance offered by active constraints can vary, but they are typically used to either guide the user along a task-specific pathway or limit the user to within a “safe” region. There are several diverse methods described within the literature for applying active constraints, and these are surveyed within this paper. The active constraint research is described and compared using a simple generalized framework, which consists of three primary processes: 1) constraint definition, 2) constraint evaluation, and 3) constraint enforcement. All relevant research approaches for each of these processes, found using search terms associated to “virtual fixture,” “active constraint” and “motion constraint,” are presented.

Active Constraints/Virtual Fixtures: A Survey

Minimally invasive surgery (MIS) is a surgical technique that offers distinct advantages in reducing pain and patients' recovery time. However, the drawback due to the lack of force and tactile feedback presents a great deal of limitations in MIS procedures. Tissue palpation, which is easily conducted during traditional open surgery to examine tissue properties and abnormalities, is not possible when performing surgery in a minimally invasive manner. This paper proposes a specially designed miniature 3-axis distal force sensor that can be used to perform tissue palpation, measuring tissue interaction forces at the tip of a surgical instrument. Relying on an optical sensing scheme, the sensor can measure forces within measurement ranges of ±3 N in axial direction and ±1.5 N in radial direction. The resolution is 0.02 N. It is compatible with laparoscopic operations and can be used to localize tissue lesions or relatively hard nodules buried under an organ's surface, which are not detectable by visual means.

Miniature 3-Axis Distal Force Sensor for Minimally Invasive Surgical Palpation

Surgical simulators present a safe and potentially effective method for surgical training, and can also be used in robot-assisted surgery for pre-and intra-operative planning. Accurate modeling of the interaction between surgical instruments and organs has been recognized as a key requirement in the development of high-fidelity surgical simulators. Researchers have attempted to model tool-tissue interactions in a wide variety of ways, which can be broadly classified as (1) linear elasticity-based, (2) nonlinear (hyperelastic) elasticity-based finite element (FE) methods, and (3) other techniques not based on FE methods or continuum mechanics. Realistic modeling of organ deformation requires populating the model with real tissue data (which are difficult to acquire in vivo) and simulating organ response in real time (which is computationally expensive). Further, it is challenging to account for connective tissue supporting the organ, friction, and topological changes resulting from tool-tissue interactions during invasive surgical procedures. Overcoming such obstacles will not only help us to model tool-tissue interactions in real time, but also enable realistic force feedback to the user during surgical simulation. This review paper classifies the existing research on tool-tissue interactions for surgical simulators specifically based on the modeling techniques employed and the kind of surgical operation being simulated, in order to inform and motivate future research on improved tool-tissue interaction models.

/pdf/modeling-of-tool-tissue-interactions-for-computer-based-877y01psk7.pdf

Modeling of tool-tissue interactions for computer-based surgical simulation: A literature review

Manual palpation of tissue and organs during a surgical procedure provides clinicians with valuable information for diagnosis and surgical planning. In present-day robotassisted minimally invasive surgery systems, lack of perceptible haptic feedback makes it challenging to detect a tumor in an organ or a calcified artery in heart tissue. This study presents an automated tissue property estimation method and a real-time graphical overlay that allow an operator to discriminate hard and soft tissues. We first evaluate experimentally the properties of an artificial tissue and compare seven possible mathematical tissue models. Self-validation as well as cross-validation confirm that the Hunt-Crossley model best describes the experimentally observed phantom tissue properties and is suitable for our purpose. Second, we present the development of a system in which the phantom tissue is palpated using a teleoperated surgical robot, and the stiffness of the Hunt-Crossly model is estimated in real time by recursive least squares. A real-time visual overlay representing tissue stiffness is created using a hue-saturation-luminance representation on a semi-transparent disc at the tissue surface. Hue depicts the stiffness at a palpated point and saturation is calculated based on distance from the point. A simple interpolation technique creates a continuous stiffness color map. In an experiment, the graphical overlay successfully shows the location of an artificial calcified artery hidden in phantom tissue.

Tissue property estimation and graphical display for teleoperated robot-assisted surgery

Background

Current teleoperated robot-assisted minimally invasive surgical systems do not take full advantage of the potential performance enhancements offered by various forms of haptic feedback to the surgeon. Direct and graphical haptic feedback systems can be integrated with vision and robot control systems in order to provide haptic feedback to improve safety and tissue mechanical property identification.



Methods

An interoperable interface for teleoperated robot-assisted minimally invasive surgery was developed to provide haptic feedback and augmented visual feedback using three-dimensional (3D) graphical overlays. The software framework consists of control and command software, robot plug-ins, image processing plug-ins and 3D surface reconstructions.



Results

The feasibility of the interface was demonstrated in two tasks performed with artificial tissue: palpation to detect hard lumps and surface tracing, using vision-based forbidden-region virtual fixtures to prevent the patient-side manipulator from entering unwanted regions of the workspace.



Conclusions

The interoperable interface enables fast development and successful implementation of effective haptic feedback methods in teleoperation. Copyright © 2011 John Wiley & Sons, Ltd.

Augmented reality and haptic interfaces for robot-assisted surgery

It is hypothesized that the lack of haptic (force and tactile) feedback presented to the surgeon is a limiting factor in the performance of teleoperated robot-assisted minimally invasive surgery. This chapter reviews the technical challenges of creating force feedback in robot-assisted surgical systems and describes recent results in creating and evaluating the effectiveness of this feedback in mock surgical tasks. In the design of a force-feedback teleoperator, the importance of hardware design choices and their relationship to controller design are emphasized. In addition, the practicality and necessity of force feedback in all degrees of freedom of the teleoperator are considered in the context of surgical tasks and the operating room environment. An alternative to direct force feedback to the surgeon’s hands is sensory substitution/augmented reality, in which graphical displays are used to convey information about the forces between the surgical instrument and the patient, or about the mechanical properties of the patient’s tissue. Experimental results demonstrate that the effectiveness of direct and graphical force feedback depend on the nature of the surgical task and the experience level of the surgeon.

Force Feedback and Sensory Substitution for Robot-Assisted Surgery

A surgical robot including an imaging system comprising at least one camera, a processor in communication with the imaging system, a manipulation system in communication with the processor, and a visual display in communication with the processor. The processor is operable to calculate a stiffness estimate for an area of an environment based on an environment model of tool-environment interaction data, create a composite image comprising a stiffness map of the stiffness estimate overlaid on an environment image from the at least one camera, and output the composite image on the visual display.

Tomonori Yamamoto

Papers

Tissue property estimation and graphical display for teleoperated robot-assisted surgery

Augmented reality and haptic interfaces for robot-assisted surgery

Force Feedback and Sensory Substitution for Robot-Assisted Surgery

Environment property estimation and graphical display

Study on eye-gaze input interface based on deep learning using images obtained by multiple cameras