BACKGROUND:
The primary limitation of telesurgery is the communication latency. Accurate and detailed data are lacking to reveal the latency effects on surgical performance; furthermore, the maximum acceptable latency in telesurgery remains unclear.
METHODS:
Sixteen medical students performed an energy dissection exercise and a needle-driving exercise on the robotic simulator dV-Trainer(®), and latencies varying between 0 and 1,000 ms with a 100-ms interval were randomly and blindly presented. Task completion time, instrument motion, and errors were automatically recorded. The difficulty, security, precision, and fluidity of manipulation were self-scored by subjects between 0 and 4 (0 the best, 2 moderate, and 4 the worst).
RESULTS:
Task completion time, motion, and errors increased gradually as latency increased. An exponential regression was fit to the mean times and motions (R (2) > 0.98). Subjective scorings of the four items were similar. The mean scores were less than 1 at delays ≤200 ms, then increased from 1 to 2 at 300-700 ms, and finally approached 3 at delays above. In both exercises, latencies ≤300 ms were judged to be safe by all and 400-500 ms were accepted by 66-75 % of subjects. Less than 20 % of subjects accepted delays ≥800 ms.
CONCLUSIONS:
The surgical performance deteriorates in an exponential way as the latency increases. The delay impact on instrument manipulation is mild at 0-200 ms, then increases from small to large at 300-700 ms, and finally becomes very large at 800-1,000 ms. Latencies ≤200 ms are ideal for telesurgery; 300 ms is also suitable; 400-500 ms may be acceptable but are already tiring; and 600-700 ms are difficult to deal with and only acceptable for low risk and simple procedures. Surgery is quite difficult at 800-1,000 ms, telementoring would be a better choice in this case.

Determination of the latency effects on surgical performance and the acceptable latency levels in telesurgery using the dV-Trainer(®) simulator.

Background
Basic telesurgical manoeuvres were conducted with signal delays.

Methods
Eight test subjects conducted four manoeuvres. Time delays of 0–1000 ms were investigated. Time to task completion and error rate were recorded in sequential delays of 0–600 ms. Additionally, blinded random delays of 0–1000 ms were studied.

Results
In the sequential trials (0–600 ms), there were no significant differences in average task time compared to zero latency. The error rate remained low despite increasing time delay, and was significantly less at 500 ms (p < 0.05). In the random trials, task time was significantly greater at delays of 500, 600, 800 and 1000 ms (p < 0.05). There were no significant differences in error rates (p = 0.252).

Conclusions
Operators are capable of performing surgical exercises at significant delays. Latent video feedback is difficult for telesurgery. Visual or virtual reality cues should be implemented to aid the operator in a high-cadence telesurgery environment. Copyright © 2006 John Wiley & Sons, Ltd.

https://www.onlinelibrary.wiley.com/doi/pdf/10.1002/rcs.99

Long-distance robotic telesurgery: a feasibility study for care in remote environments.

When we manipulate an object, grip force is adjusted in anticipation of the mechanical consequences of hand motion (i.e., load force) to prevent the object from slipping. This predictive behavior is assumed to rely on an internal representation of the object dynamic properties, which would be elaborated via visual information before the object is grasped and via somatosensory feedback once the object is grasped. Here we examined this view by investigating the effect of delayed visual feedback during dextrous object manipulation. Adult participants manually tracked a sinusoidal target by oscillating a handheld object whose current position was displayed as a cursor on a screen along with the visual target. A delay was introduced between actual object displacement and cursor motion. This delay was linearly increased (from 0 to 300 ms) and decreased within 2-min trials. As previously reported, delayed visual feedback altered performance in manual tracking. Importantly, although the physical properties of the object remained unchanged, delayed visual feedback altered the timing of grip force relative to load force by about 50 ms. Additional experiments showed that this effect was not due to task complexity nor to manual tracking. A model inspired by the behavior of mass-spring systems suggests that delayed visual feedback may have biased the representation of object dynamics. Overall, our findings support the idea that visual feedback of object motion can influence the predictive control of grip force even when the object is grasped.

https://www.physiology.org/doi/pdf/10.1152/jn.00174.2010

Delayed Visual Feedback Affects Both Manual Tracking and Grip Force Control When Transporting a Handheld Object

Long-term human space exploration will require contingencies for emergency medical procedures including some capability to perform surgery. The ability to perform minimally invasive surgery (MIS) would be an important capability. The use of small incisions reduces surgical risk, but also eliminates the ability of the surgeon to view and touch the surgical environment directly. Robotic surgery, or telerobotic surgery, may provide emergency surgical care in remote or harsh environments such as space flight, or extremely forward environments such as battlefields. However, because current surgical robots are large and require extensive support personnel, their implementation has remained limited in forward environments, and they would be difficult, or impossible, to use in space flight or on battlefields. This paper presents experimental analysis of miniature fixed-base and mobile in vivo robots to support MIS surgery in remote and harsh environments. The objective is to develop wireless imaging and task-assisting robots that can be placed inside the abdominal cavity during surgery. Such robots will provide surgical task assistance and enable an on-site or remote surgeon to view the surgical environment from multiple angles. This approach is applicable to long-duration space flight, battlefield situations, and for traditional medical centers and other remote surgical locations.

/pdf/miniature-in-vivo-robots-for-remote-and-harsh-environments-iqy9hxjxs3.pdf

Miniature in vivo Robots for Remote and Harsh Environments

Fully autonomous applications of modern robotic systems are still constrained by limitations in sensory data processing, scene interpretation, and automated reasoning. However, their use as assistive devices for people with upper-limb disabilities has become possible with recent advances in âsoft roboticsâ, that is, interaction control, physical human-robot interaction, and reflex planning. In this context, impedance and reflex-based control has generally been understood to be a promising approach to safe interaction robotics.

To create semi-autonomous assistive devices, we propose a decision-and-control architecture for hand-arm systems with âsoft roboticsâ capabilities, which can then be used via human-machine interfaces HMIs. We validated the functionality of our approach within the BrainGate2 clinical trial, in which an individual with tetraplegia used our architecture to control a robotic hand-arm system under neural control via a multi-electrode array implanted in the motor cortex. The neuroscience results of this research have previously been published by Hochberg et al.

In this paper we present our assistive decision-and-control architecture and demonstrate how the semi-autonomous assistive behavior can help the user. In our framework the robot is controlled through a multi-priority Cartesian impedance controller and its behavior is extended with collision detection and reflex reaction. Furthermore, virtual workspaces are added to ensure safety. On top of this we employ a decision-and-control architecture that uses sensory information available from the robotic system to evaluate the current state of task execution. Based on a set of available assistive skills, our architecture provides support in object interaction and manipulation and thereby enhances the usability of the robotic system for use with HMIs. The goal of our development is to provide an easy-to-use robotic system for people with physical disabilities and thereby enable them to perform simple tasks of daily living. In an exemplary real-world task, the participant was able to serve herself a beverage autonomously for the first time since her brainstem stroke, which she suffered approximately 14 years prior to this research.

An assistive decision-and-control architecture for force-sensitive hand-arm systems driven by human-machine interfaces

Telerobotic surgery is ideally suited for remote applications in which the instrument control console is stationed separately from the end-effectors at the patient’s bedside. However, if the distance between the console and the patient is great enough, a lag effect or latency between end-effector manipulation and the depicted image leads to alterations in movement patterns. The purpose of this study was to determine the effect of visual delay on surgical task performance. At an endoscopic skill station, an analogue delay device was interposed between the surgical field and monitor to delay the transmission of visual information, thus mimicking the distance effect of data transmission. Three surgeons with similar laparoscopic experience participated in the laparoscopic knot tying portion of the study, and seven residents participated in the accuracy and dexterity tasks. The time to complete a single throw was recorded in seconds after adding consecutively increasingly time delay in 50 ms increments. Similar time delay increments were added for the accuracy and dexterity tasks, which involved passing a needle through two adjacent circles and passing a small cylinder through a larger one to reproduce two-handed coordination and spatial resolution. Data were presented as the median time to complete each task. For all three tasks, an incremental increase in time delay was associated with a significant (p < 0.001) increase in the time to complete the task. For dexterity, a statistically significant (p ≤ 0.05) delay was identified at 0.25 s of delay from control values without delay. A move-and-wait strategy was gradually adopted up to 0.4 s of visual delay. Compensation for visually delayed image perception occurs on several levels. Initial adaptations include slower end-effector manipulation; late adaptive changes include a move-and-wait strategy. Increased time to perform surgical maneuvers as well as diminished accuracy, diminished dexterity, and increasing fatigue represent additional performance encumbrances evoked by visual time delay. The nuances of both human and digital compensatory mechanisms for visual time delay must be defined and enhanced to maximize the potential for telerobotic surgical applications.

M. J. Wade

Papers

The effect of delayed visual feedback on telerobotic surgery.