https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Policy search methods can allow robots to learn control policies for a wide range of tasks, but practical applications of policy search often require hand-engineered components for perception, state estimation, and low-level control. In this paper, we aim to answer the following question: does training the perception and control systems jointly end-to-end provide better performance than training each component separately? To this end, we develop a method that can be used to learn policies that map raw image observations directly to torques at the robot's motors. The policies are represented by deep convolutional neural networks (CNNs) with 92,000 parameters, and are trained using a guided policy search method, which transforms policy search into supervised learning, with supervision provided by a simple trajectory-centric reinforcement learning method. We evaluate our method on a range of real-world manipulation tasks that require close coordination between vision and control, such as screwing a cap onto a bottle, and present simulated comparisons to a range of prior policy search methods.

/pdf/end-to-end-training-of-deep-visuomotor-policies-1vqd5uo50d.pdf

End-to-end training of deep visuomotor policies

Here we present deep-learning techniques for healthcare, centering our discussion on deep learning in computer vision, natural language processing, reinforcement learning, and generalized methods. We describe how these computational techniques can impact a few key areas of medicine and explore how to build end-to-end systems. Our discussion of computer vision focuses largely on medical imaging, and we describe the application of natural language processing to domains such as electronic health record data. Similarly, reinforcement learning is discussed in the context of robotic-assisted surgery, and generalized deep-learning methods for genomics are reviewed.

A guide to deep learning in healthcare.

A surgical tool system includes a surgical tool for cutting and stapling tissue and a tactile feedback system integrated onto a handle of the tool that generates relevant feedback in at least the form of haptic effects to the user. The tactile feedback alerts the user of tissue properties, i.e., the type of tissue or other structures located within the jaws of the stapler, whether the quantity or thickness of tissue located within the jaws of the stapler is appropriate for the selected stapler cartridge, whether the proper length of staples has been fired based on the length of tissue located in the jaws, whether a blood vessel is located within the jaws of the stapler, whether the stapling process has successfully sealed the tissue located within the jaws of the stapler, the position of the cutting element, and/or when the stapling procedure or firing cycle is completed.

Surgical stapler having haptic feedback

In this work we resolve the long-outstanding problem of how to effectively train recurrent neural networks (RNNs) on complex and difficult sequence modeling problems which may contain long-term data dependencies Utilizing recent advances in the Hessian-free optimization approach (Martens, 2010), together with a novel damping scheme, we successfully train RNNs on two sets of challenging problems First, a collection of pathological synthetic datasets which are known to be impossible for standard optimization approaches (due to their extremely long-term dependencies), and second, on three natural and highly complex real-world sequence datasets where we find that our method significantly outperforms the previous state-of-the-art method for training neural sequence models: the Long Short-term Memory approach of Hochreiter and Schmidhuber (1997) Additionally, we offer a new interpretation of the generalized Gauss-Newton matrix of Schraudolph (2002) which is used within the HF approach of Martens

/pdf/learning-recurrent-neural-networks-with-hessian-free-1ijyy1rxv3.pdf

Learning Recurrent Neural Networks with Hessian-Free Optimization

Tying suture knots is a time-consuming task performed frequently during Minimally Invasive Surgery (MIS). Automating this task could greatly reduce total surgery time for patients. Current solutions to this problem replay manually programmed trajectories, but a more general and robust approach is to use supervised machine learning to smooth surgeon-given training trajectories and generalize from them. Since knottying generally requires a controller with internal memory to distinguish between identical inputs that require different actions at different points along a trajectory, it would be impossible to teach the system using traditional feedforward neural nets or support vector machines. Instead we exploit more powerful, recurrent neural networks (RNNs) with adaptive internal states. Results obtained using LSTM RNNs trained by the recent Evolino algorithm show that this approach can significantly increase the efficiency of suture knot tying in MIS over preprogrammed control.

/pdf/a-system-for-robotic-heart-surgery-that-learns-to-tie-knots-2wx0u3llje.pdf

A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks

Minimally invasive robotic surgery systems has entered daily practice in dedicated clinical centers. Especially heart surgery profits from this new technique, due to a higher accuracy compared to conventional endoscopic interventions. Nevertheless some drawbacks have restricted a broader acceptance of these devices. The most urgent issues are lack of haptic feedback and prolonged operation time. Our research project tackles both topics, on the one hand by adding high fidelity force-feedback, on the other hand by automating recurrent manipulation tasks. These features have been integrated into the Endo[PA]R (endoscopic partially-autonomous robot) system, an open evaluation platform for robotic surgery. The setup comprises two low-payload robots equipped with sensorized surgical instruments and a third robot carrying a stereo camera system. Trocar kinematics, enabling surgical manipulation through small incisions, has been implemented for all robotic arms. In order to ensure operation close to reality, a thorax and heart phantom for surgical training was used in the experiments. Stereo vision is provided via a head-mounted display and force-reflective input devices are employed for user interaction. The system was evaluated by surgeons and it was capable of performing autonomous knot-tying.

/pdf/the-endo-pa-r-system-for-minimally-invasive-robotic-surgery-1z71m1b6fq.pdf

The Endo[PA]R system for minimally invasive robotic surgery

The implementation of telemanipulator systems for cardiac surgery enabled heart surgeons to perform delicate minimally invasive procedures with high precision under stereoscopic view. At present, commercially available systems do not provide force-feedback or Cartesian control for the operating surgeon. The lack of haptic feedback may cause damage to tissue and can cause breaks of suture material. In addition, minimally invasive procedures are very tiring for the surgeon due to the need for visual compensation for the missing force feedback. While a lack of Cartesian control of the end effectors is acceptable for surgeons (because every movement is visually supervised), it prevents research on partial automation. In order to improve this situation, we have built an experimental telemanipulator for endoscopic surgery that provides both force-feedback (in order to improve the feeling of immersion) and Cartesian control as a prerequisite for automation. In this article, we focus on the inclusion of force feedback and its evaluation. We completed our first bimanual system in early 2003 (EndoPAR Endoscopic Partial Autonomous Robot). Each robot arm consists of a standard robot and a surgical instrument, hence providing eight DOF that enable free manipulation via trocar kinematics. Based on the experience with this system, we introduced an improved version in early 2005. The new ARAMIS system (Autonomous Robot Assisted Minimally Invasive Surgery) has four multi-purpose robotic arms mounted on a gantry above the working space. Again, the arms are controlled by two force-feedback devices, and 3D vision is provided. In addition, all surgical instruments have been equipped with strain gauge force sensors that can measure forces along all translational directions of the instrument's shaft. Force-feedback of this system was evaluated in a scenario of robotic heart surgery, which offers an impression very similar to the standard, open procedures with high immersion. It enables the surgeon to palpate arteriosclerosis, to tie surgical knots with real suture material, and to feel the rupture of suture material. Therefore, the hypothesis that haptic feedback in the form of sensory substitution facilitates performance of surgical tasks was evaluated on the experimental platform described in the article (on the EndoPAR version). In addition, a further hypothesis was explored: The high fatigue of surgeons during and after robotic operations may be caused by visual compensation due to the lack of force-feedback (Thompson, J., Ottensmeier, M., & Sheridan, T. 1999. Human Factors in Telesurgery, Telmed Journal, 5 (2) 129--137.).

Haptic Feedback in a Telepresence System for Endoscopic Heart Surgery

A popular method for an easy and also flexible programming of robots is learning by demonstration. An intelligent controller learns a task from several examples carried out by an experienced user. Afterwards, the task can be adapted to new, formerly unknown environments. One particular challenge arising with this technique is generalization of demonstrations in order to get a generic description of the task. In this paper a new methodology for solving this problem is proposed. The main part of the algorithm exploits principles known from fluid dynamics.

/pdf/adaptive-control-for-human-robot-skilltransfer-trajectory-3kssfp46jj.pdf

Adaptive Control for Human-Robot Skilltransfer: Trajectory Planning Based on Fluid Dynamics

Since the publication of the first paper on NOTES, a dramatic increase of activities in this new surgical field is to be observed. However, a "pure" NOTES operation does not yet exist. Only NOTES-like operations have been proposed so far. One of the essential barriers is the limited intraoperative performance due to the lack of suitable mechatronic platforms. Some solutions have already been suggested but they are still far from having any practical impact upon the development of more advanced NOTES procedures. The "Highly Versatile Single Port System", a two-armed device with two manipulators and a semi-flexible telescope, was developed to overcome these drawbacks. A transsigmoid cholecystectomy on a NOTES simulator (the ELITE) could be accomplished successfully in 75 minutes. We could therefore show that the HVSPS is in principle applicable for such a NOTES procedure. However, before these mechatronic support systems can be used under clinical conditions, required human machine interface and extended intelligence have to be solved.

Hermann Mayer

Papers

A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks

The Endo[PA]R system for minimally invasive robotic surgery

Haptic Feedback in a Telepresence System for Endoscopic Heart Surgery

Adaptive Control for Human-Robot Skilltransfer: Trajectory Planning Based on Fluid Dynamics

The mechatronic support system "HVSPS" and the way to NOTES.