Let's read! We will often find out this sentence everywhere. When still being a kid, mom used to order us to always read, so did the teacher. Some books are fully read in a week and we need the obligation to support reading. What about now? Do you still love reading? Is reading only for you who have obligation? Absolutely not! We here offer you a new book enPDFd the perception of the visual world to read.

The Perception of the Visual World. By James J. Gibson. U.S.A.: Houghton Mifflin Company, 1950 (George Allen & Unwin, Ltd., London). Price 35s.

Minimally invasive surgery represents one of the main evolutions of surgical techniques aimed at providing a greater benefit to the patient. However, minimally invasive surgery increases the operative difficulty since the depth perception is usually dramatically reduced, the field of view is limited and the sense of touch is transmitted by an instrument. However, these drawbacks can currently be reduced by computer technology guiding the surgical gesture. Indeed, from a patient’s medical image (US, CT or MRI), Augmented Reality (AR) can increase the surgeon’s intra-operative vision by providing a virtual transparency of the patient. AR is based on two main processes: the 3D visualization of the anatomical or pathological structures appearing in the medical image, and the registration of this visualization on the real patient. 3D visualization can be performed directly from the medical image without the need for a pre-processing step thanks to volume rendering. But better results are obtained with surface rendering after organ and pathology delineations and 3D modelling. Registration can be performed interactively or automatically. Several interactive systems have been developed and applied to humans, demonstrating the benefit of AR in surgical oncology. It also shows the current limited interactivity due to soft organ movements and interaction between surgeon instruments and organs. If the current automatic AR systems show the feasibility of such system, it is still relying on specific and expensive equipment which is not available in clinical routine. Moreover, they are not robust enough due to the high complexity of developing a real-time registration taking organ deformation and human movement into account. However, the latest results of automatic AR systems are extremely encouraging and show that it will become a standard requirement for future computer-assisted surgical oncology. In this article, we will explain the concept of AR and its principles. Then, we will review the existing interactive and automatic AR systems in digestive surgical oncology, highlighting their benefits and limitations. Finally, we will discuss the future evolutions and the issues that still have to be tackled so that this technology can be seamlessly integrated in the operating room.

Augmented reality in laparoscopic surgical oncology.

The status of augmented reality in laparoscopic surgery as of 2016.

Computer-assisted oral and maxillofacial surgery (OMS) has been rapidly evolving since the last decade. State-of-the-art surgical navigation in OMS still suffers from bulky tracking sensors, troublesome image registration procedures, patient movement, loss of depth perception in visual guidance, and low navigation accuracy. We present an augmented reality navigation system with automatic marker-free image registration using 3-D image overlay and stereo tracking for dental surgery. A customized stereo camera is designed to track both the patient and instrument. Image registration is performed by patient tracking and real-time 3-D contour matching, without requiring any fiducial and reference markers. Real-time autostereoscopic 3-D imaging is implemented with the help of a consumer-level graphics processing unit. The resulting 3-D image of the patient's anatomy is overlaid on the surgical site by a half-silvered mirror using image registration and IP-camera registration to guide the surgeon by exposing hidden critical structures. The 3-D image of the surgical instrument is also overlaid over the real one for an augmented display. The 3-D images present both stereo and motion parallax from which depth perception can be obtained. Experiments were performed to evaluate various aspects of the system; the overall image overlay error of the proposed system was 0.71 mm.

Augmented Reality Navigation With Automatic Marker-Free Image Registration Using 3-D Image Overlay for Dental Surgery

Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display

A 3-D augmented reality navigation system using autostereoscopic images was developed for MRI-guided surgery. The 3-D images are created by employing an animated autostereoscopic image, integral videography (IV), which provides geometrically accurate 3-D spatial images and reproduces motion parallax without using any supplementary eyeglasses or tracking devices. The spatially projected 3-D images are superimposed onto the surgical area and viewed via a half-slivered mirror. A fast and accurate spatial image registration method was developed for intraoperative IV image-guided therapy. Preliminary experiments showed that the total system error in patient-to-image registration was 0.90 ± 0.21 mm, and the procedure time for guiding a needle toward a target was shortened by 75%. An animal experiment was also conducted to evaluate the performance of the system. The feasibility studies showed that augmented reality of the image overlay system could increase the surgical instrument placement accuracy and reduce the procedure time as a result of intuitive 3-D viewing.

3-D Augmented Reality for MRI-Guided Surgery Using Integral Videography Autostereoscopic Image Overlay

Intraoperative tumor segmentation and volume measurement in MRI-guided glioma surgery for tumor resection rate control.

A three-dimensional (3D) image overlay navigation system has been developed for open MRI guided surgery. The images are created by employing animated autostereoscopic image of integral videography (IV), which provides geometrically accurate 3D images and reproduce motion parallax without using any supplementary glasses or tracking devices. The spatially projected 3D images are superimposed onto the patient and are viewed via a half-slivered mirror. We developed a spatial image registration method for intraoperative IV image-guided diagnosis and therapy. Two sets of preliminary experiments showed that the total system error was 0.90 ± 0.21mm in patient-to-image registration and the procedure time of guiding a needle toward a target was shortened to 24%. The augmented reality of the image overlay system is possible to increase the surgical instrument placement accuracy and also to reduce the procedure time as the result of intuitive 3D viewing.

Surgical navigation of integral videography image overlay for open MRI-Guided glioma surgery

We propose an automatic markerless registration method for Integral Videography (IV) overlay navigation system that directly matches a video image of the patient to a 2-D rendering of a surface model extracted from the 3-D surface model image. IV overlay system is integrated into a surgical navigation system to superimpose a real three-dimensional (3-D) autostereoscopic image onto the patient via a half-silvered mirror. This registration technique doesn’t require the setting of markers, which is invasive and extends the duration of the procedure. Accuracy measurements were performed using a triangular-prism-shaped model. The mean Target Registration Error (TRE) was 1.6 ± 0.6 mm. We also applied our method to a human face registration and obtained successful results, although some noises were added to the video image. The presented method is more suitable for clinical applications of IV image overlay than previous methods since the presented method is sufficiently accurate, robust and more convenient for surgeon.

Integral Videography Overlay Navigation System Using Mutual Information-Based Registration

We employed Fuzzy Connectedness method to perform an intraoperative segmentation [1]. We limited the region to be analyzed, so the processing time became less than ten seconds. The accuracy of the presented method was validated in nine patients with glioma. Goodness-of-match between the manual and automatic segmentation was evaluated by two established measurements of segmentation accuracy, Percent Match (PM) and Dice Similarity Coefficient (DSC).

Takashi Inomata

Papers

3-D Augmented Reality for MRI-Guided Surgery Using Integral Videography Autostereoscopic Image Overlay

Intraoperative tumor segmentation and volume measurement in MRI-guided glioma surgery for tumor resection rate control.

Surgical navigation of integral videography image overlay for open MRI-Guided glioma surgery

Integral Videography Overlay Navigation System Using Mutual Information-Based Registration

Intraoperative segmentation of brain tumors for open MRI guided glioma surgery