This is an excellent overview of the pros and cons of surgical VR techniques. In the Netherlands, it is concluded that training scenarios for complicated endoscopic procedures are lacking. A cultural shift in the teaching hospitals is thus needed both for trainers and trainees.[94] The authors also focus on the fact that so far it has not been proven that a virtual environment can improve neurosurgical training. The question “Why is that so?” is not fully answered! Limited dexterity will result in neurosurgical trainees not developing their skills to an extent that will convert them from novices into experienced surgeons.[89–91] Most importantly, however, all surgeons need to be able to reflect over how they are performing during operative procedures if they are to become true experts.[92] Moving into the topic “endoscopic surgery,” we also change our working space in many ways.[88] Most importantly, we change our normal working 3D space into a 2D space. Thereby we have to learn new ways of obtaining a 3D perception. Most literature on surgical VR techniques is dealing with multi-port endoscopy, where surgeons are working in an artificial, distended operative field. This is in contrast to neurosurgery, where only one endoscopic port is used. Models of removing gall bladders were among the first VR scenarios. While training with VR-simulators, there is a significant difference in the number of mistakes and speed between experienced and novice surgeons.[87] This fact cannot simply be transfered to the learning of endoscopy in neurosurgery. In endoscopic neurosurgery, using a single endoscope to both view and manipulate instruments in narrow spaces under the brain with very limited degrees of finger/hand movements makes a significant difference.[89] In real surgical life, sensing is a key point for neurosurgeons. For spatio/temporal skills, using the sense of touch is an important skill to learn and master. For technical reasons, haptics are most often not included in the VR systems on the market.[95] Verbal teaching of haptics is a challenging task. The range of haptic sensitivity varies from person to person, and possibly also from dominant to non-dominant hand in the same person. Bring a brain retractor into the field and demonstrate to your trainee how gentle your retraction of the brain is. How can the trainee sense what you are doing?[89,93] To the best of my knowledge, the topic of “how to learn to use a brain spatula” has never been introduced in neurosurgical training, not even in the present paper.No one mentions the fact that the trainee learns only how to use a spatula by watching his mentor in no way can the pressure-sensing of the spatula on the brain be transferred from trainer to trainee in the present operating room situations. We have therefore developed a Virtual Brain model using haptic feedback to develop different pressures of the retractor, and thus different deformations of the brain.[93] Now we can teach trainees how the true pressure equals the monitored pressure and the brain shape/deformations are correlated.[86] The neurosurgical VR training is thus different from training in conventional multi-port endoscopy. Neurosurgeons have to use VR systems that develop dexterity and transfer it into an automatic dexterity, whereby haptics are needed in the coming neurosurgical VR models.

Virtual reality training in neurosurgery: Review of current status and future applications

Endoscopic procedures form part of routine clinical practice for minimally invasive examinations and interventions. While they are beneficial for the patient, reducing surgical trauma and making convalescence times shorter, they make orientation and manipulation more challenging for the physician, due to the limited field of view through the endoscope. However, this drawback can be reduced by means of medical image processing and computer vision, using image stitching and surface reconstruction methods to expand the field of view. This paper provides a comprehensive overview of the current state of the art in endoscopic image stitching and surface reconstruction. The literature in the relevant fields of application and algorithmic approaches is surveyed. The technological maturity of the methods and current challenges and trends are analyzed.

Stitching and Surface Reconstruction From Endoscopic Image Sequences: A Review of Applications and Methods

Surgeries of the skull base require accuracy to safely navigate the critical anatomy. This is particularly the case for endoscopic endonasal skull base surgery (ESBS) where the surgeons work within millimeters of neurovascular structures at the skull base. Today's navigation systems provide approximately 2 mm accuracy. Accuracy is limited by the indirect relationship of the navigation system, the image and the patient. We propose a method to directly track the position of the endoscope using video data acquired from the endoscope camera. Our method first tracks image feature points in the video and reconstructs the image feature points to produce 3D points, and then registers the reconstructed point cloud to a surface segmented from preoperative computed tomography (CT) data. After the initial registration, the system tracks image features and maintains the 2D-3D correspondence of image features and 3D locations. These data are then used to update the current camera pose. We present a method for validation of our system, which achieves submillimeter (0.70 mm mean) target registration error (TRE) results.

https://www.cs.jhu.edu/~hwang/papers/TMI2011.pdf

A System for Video-Based Navigation for Endoscopic Endonasal Skull Base Surgery

3D tracking and positioning of surgical instruments is an indispensable part of virtual Surgery training system, because it is the unique interface for trainee to communicate with virtual environment. A suit of 3D tracking and positioning of surgical instruments based on stereoscopic vision is proposed. It can capture spatial movements of simulated surgical instrument in real time, and provide 6 degree of freedom information with the absolute error of less than 1 mm. The experimental results show that the 3D tracking and positioning of surgical instruments is highly accurate, easily operated, and inexpensive. Combining with force sensor and embedded acquisition device, this 3D tracking and positioning method can be used as a measurement platform of physical parameters to realize the measurement of soft tissue parameters.

https://www.researchgate.net/profile/Zhaoliang_Duan/publication/220590733_3D_Tracking_and_Positioning_of_Surgical_Instruments_in_Virtual_Surgery_Simulatio/links/0f31753cc8d40978b8000000.pdf

3D Tracking and Positioning of Surgical Instruments in Virtual Surgery Simulation

Virtual endoscopy is the computerized creation of images depicting the inside of patient anatomy reconstructed in a virtual reality environment. It permits interactive, noninvasive, 3-dimensional visual inspection of anatomical cavities or vessels. This can aid in diagnostics, potentially replacing an actual endoscopic procedure, and help in the preparation of a surgical intervention by bridging the gap between plain 2-dimensional radiologic images and the 3-dimensional depiction of anatomy during actual endoscopy. If not only the endoscopic vision but also endoscopic handling, including realistic haptic feedback, is simulated, virtual endoscopy can be an effective training tool for novice surgeons. In neurosurgery, the main fields of the application of virtual endoscopy are third ventriculostomy, endonasal surgery, and the evaluation of pathologies in cerebral blood vessels. Progress in this very active field of research is achieved through cooperation between the technical and the medical communities. While the technology advances and new methods for modeling, reconstruction, and simulation are being developed, clinicians evaluate existing simulators, steer the development of new ones, and explore new fields of application. This review introduces some of the most interesting virtual reality systems for endoscopic neurosurgery developed in recent years and presents clinical studies conducted either on areas of application or specific systems. In addition, benefits and limitations of single products and simulated neuroendoscopy in general are pointed out.

Virtual endoscopy in neurosurgery: a review.

Purpose
Virtual endoscopy has already proven its benefit for pre-operative planning of endoscopic pituitary surgery. The translation of such a system into the operating room is a logical consequence, but only a few general intra-operative image guided systems providing virtual endoscopic images have been proposed so far. A discussion of related visualization and interaction problems occurring during sinus and pituitary surgery is still missing.

https://www.vrvis.at/publications/pdfs/PB-VRVis-2009-010.pdf

Leslie Holton

Papers

Intra-operative virtual endoscopy for image guided endonasal transsphenoidal pituitary surgery.