Augmented Reality to Improve Surgical Simulation: Lessons Learned Towards the Design of a Hybrid Laparoscopic Simulator for Cholecystectomy

TL;DR: The proposed AR simulator satisfies all the initial specifications in terms of anatomical appearance, modularity, reusability, minimization of spare parts cost, and ability to record surgical errors and to track in real-time the Calot's triangle and the laparoscope.

Abstract: Hybrid surgical simulators based on augmented reality (AR) solutions benefit from the advantages of both the box trainers and the virtual reality simulators. This paper reports on the results of a long development stage of a hybrid simulator for laparoscopic cholecystectomy that integrates real and the virtual components. We first outline the specifications of the AR simulator and then we explain the strategy adopted for implementing it based on a careful selection of its simulated anatomical components, and characterized by a real-time tracking of both a target anatomy and of the laparoscope. The former is tracked by means of an electromagnetic field generator, while the latter requires an additional camera for video tracking. The new system was evaluated in terms of AR visualization accuracy, realism, and hardware robustness. Obtained results show that the accuracy of AR visualization is adequate for training purposes. The qualitative evaluation confirms the robustness and the realism of the simulator. In conclusion, the proposed AR simulator satisfies all the initial specifications in terms of anatomical appearance, modularity, reusability, minimization of spare parts cost, and ability to record surgical errors and to track in real-time the Calot's triangle and the laparoscope. Thus, the proposed system could be an effective training tool for learning the task of identification and isolation of Calot's triangle in laparoscopic cholecystectomy. Moreover, the presented strategy could be applied to simulate other surgical procedures involving the task of identification and isolation of generic tubular structures, such as blood vessels, biliary tree, and nerves, which are not directly visible.

Summary

Introduction

• Several medical simulators exist today on the market mainly due to the increasing demand for minimally invasive surgical (MIS) procedures and the increasing concern on patient safety [1].
• According to the physical-virtual simulation spectrum proposed by Samsun Lampotang et al. [17], AR simulation is a form of mixed simulation which combines physical simulation (such as laparoscopic equipment or/and mannequins) and VR simulation within a unique simulation environment.
• Concerning the non-commercial AR laparoscopic simulators, Lahanas et al. have described an AR simulator for MIS basic skills (e.g., navigation, peg transfer and clipping).
• The methodologies for AR implementation and laparoscope tracking and the results of the experimental tests.

II. Material and Methods

• The basic design specifications for their hybrid LC simulator are: realistic anatomical appearance, modularity, reusability, minimization of spare parts cost, AR visualization, ability to signal surgical errors and to track in real time both Calot’ s triangle structures and laparoscope.
• The simulated anatomy is chosen taking into account all the anatomical structures that could be either seen or touched during the execution of the procedure.
• In particular, BT and AT, which are sensorized and thus expensive, are designed to be reusable.
• The proto- type is designed to enable the EM tracking of the Calot’s triangle structures.
• The CAD project integrates both 3D anatomical models and 3D models of the required electronic accessories (Fig. 1).

C. Fabrication of the Real Simulator Components

• Liver, gallbladder, pancreas, ab-dominal aorta, esophagus-stomachduodenum., also known as Patient specific models.
• The strategy used for the patient specific models’ fabrication consists of four 4 main steps: extraction of the 3D models of the target organs starting from CT images; molds designing in the 3D CAD software; molds manufacturing with a 3D printer (Dimension Elite 3D Printer); casting of the chosen materials into the molds.
• Table II shows the chosen anatomical variants of CA and of CY and the number of nitinol tubes used for the fabrication of each portion of AT and BT.
• . anatomical relationships of AT with the liver, gallbladder and aorta:.
• In detail, the upper and lower ends of the BT and AT are equipped with a male connector whose function can be either purely mechanical or mechanical- electrical.

D. Overview of the AR simulator

• The system is set up to track in real time both the laparoscope and the Calot’s triangle sensorized real components and to co- herently visualize the virtual content on the real laparoscopic image (Fig. 8).
• The laparoscope and the USB camera are moved in 20 different positions; for each pair of poses “i”, the relative motion of each coordinate frame (C and L) is recorded and stored in the two homogeneous transformation matrices: Ai and Bi. In Fig. 9, Ai denotes the motion between the poses Ti and Ti+1 .
• The application was created following the same logic of the AR software framework previ- ously developed and presented in [39].
• The experiments were performed using the set-up described in section C without covering the target structures with the connective tissue; at the beginning of each experiment the position of the laparoscope was fixed.
• The Wilcoxon signed-ranks test was used to determine the significance of the responses to each item evaluating if the operators were significantly more likely to agree or disagree with each of the statements.

III. RESULTS

• The experimental results demonstrate that the accuracy in AR visualization is adequate for training purposes as qualitatively shown in Fig. 12.
• Fig. 13 shows the mean error, maximum error and standard deviation of the TVE3D calculated for each tract.
• B. Evaluation of Simulator Realism and Its Robustness.
• The surgeons positively evaluate the realism of the connective tissue and, the usefulness of both acoustic functionality and AR scene for the training.: a median score of 4 was obtained for all the items as showed by Table IV.
• Moreover, no damages to the simulator components and their connections were detected after the trials.

IV. DISCUSSION

• The obtained results confirm the feasibility of the proposed strategy to track the laparoscope and the Calot’s triangle to co- herently visualize the latter in AR mode.
• The obtained accuracy is mainly affected by four sources of error: the inherent accuracy of EM tracking paired with the field distortions arising from the environment; the difficulties in positioning the EM sensors at the tubular structures centerline [17]; the interpolation error when drawing each virtual tract; and the errors accumulated during the system calibration.
• The surgeons confirmed, the realism of the simulator in reproducing the interaction between the surgical instruments and the organs, the arteries and the biliary tree.
• After the early stages of the learning curve, the AR visual- ization of the Calot’s triangle should be turned on only in case of an occurring surgical damage to the sensorized structure.
• - Compare their simulator and current methods of medical training demonstrating its efficiency.

V. CONCLUSION

• An advanced version of an AR simulator for la- paroscopic cholecystectomy is presented.
• Tracking methods for localizing the laparoscope and the tubular anatomical structures are described.
• Moreover, a strategy to reconstruct the shape of the Calot’s triangle structures is presented, as well as a specific calibration procedure that allows the laparoscope to be freely moved maintaining the geometric coherence of the AR scene.
• Further studies will focus on demonstration of the effective- ness, of the validity and of the appropriateness of the simulator as a training tool for novices.
• This is the first example of a hy- brid AR simulator that offers a deep integration between real and virtual components.

Figures

Fig. 9. Set-up of hand-eye calibration.

Fig. 8. Transformation (represented by dashed line) and Coordinate frames.

Fig. 6. Circuit for acoustic functionality: a) Simple electric circuit sig- naling the damages to AT and/or BT walls b) Equivalent electric scheme.

Fig. 7. Laparoscope equipped with the block and the USB camera.

TABLE I SELECTION OF MIXTURES FOR REALIZATION OF ANATOMICAL REPLICAS

Fig. 2. Anatomical replicas: Liver, Esophagus-Stomach- Duodenum, Pancreas, Aorta; Gallbladder; The zoomed detail shows the female connector inside the liver

Fig. 4. Synthetic connective tissue.

Fig. 5. Assembly and exploded view of the 3D models of the simulator: a) Assembly view of the 3D models of the simulator; the zoomed detail shows the AT and BT not clearly visible because covered by connective tissue in whole assembly b) Exploded view of the 3D models of the simulator to show the connection strategy.

Fig. 11. Experimental test: two expert surgeons before performing the surgical test. A surgeon manages the laparoscope, while the operating surgeon holds a surgical hook (blue shaft).

Fig. 12. AR frames for each one of the twelve positions of laparoscope in both configurations: lifted liver and gallbladder in lateral traction. The virtual tubes mimic real tracts of AT and BT, d is the distance between target and laparoscope expressed in mm.

TABLE II NUMBER OF NITINOL TUBES

Fig. 3. Anatomical variants of AT and BT: a) CA originating from RHA (standard configuration); b) CY joining with CHD (standard configura- tion); c) double CA originating from RHA; d) accessory duct (AD) joining CHD. The red cylinders represent the sensor’ position.

Fig. 14. AR alignment accuracy, measured as TVE3D, over each one of 12 positions of laparoscope for four tracts in corresponding to: a) lifted liver; b) gallbladder in lateral traction. The graph reports mean error, maximum error and standard deviation.

Fig. 13. AR alignment accuracy, measured as TVE3D, over each one of the four tracts for all laparoscope positions in corresponding to: a) lifted liver; b) gallbladder in lateral traction. The graph reports mean error, maximum error and standard deviation.

Fig. 1. CAD project of the simulator real components: a) 3D model of whole simulator; b-i) 3D model of each component of simulator: b) Esophagus-Stomach-Duodenum; c) Connective tissue, d) Gallbladder;

Fig. 10. Evaluation of TVE2D : a) An AR frame acquired by laparoscope. b) Segmentation of the real and the corresponding virtual CY-CD tract, c) Extraction and comparison of two centerlines.

Full text

Augmented Reality to Improve Surgical Simulation: Lessons Learned Towards the Design of
a Hybrid Laparoscopic Simulator for Cholecystectomy
Rosanna M. Viglialoro, Nicola Esposito, Sara Condino, Fabrizio Cutolo, Member, IEEE, Simone
Guadagni, Marco Gesi, Mauro Ferrari, and Vincenzo Ferrari, Member, IEEE
R. M. Viglialoro is with the EndoCAS Center, Department of Transla- tional Research and New Technologies in Medicine and Surgery, Univer- sity
of Pisa, Pisa 56126, Italy (e-mail: rosanna.viglialoro@endocas.org).
N. Esposito, S. Condino, and F. Cutolo are with the EndoCAS Cen- ter, Department of Translational Research and New Technologies in
Medicine and Surgery, University of Pisa.
S. Guadagni is with the Department of General Surgery Unit, Cisanello University Hospital AOUP.
M. Gesi is with the Department of Translational Research on New Technologies in Medicine and Surgery, University of Pisa and also with the
Center Rehabilitative Medicine “Sport and Anatomy, University of Pisa.
M. Ferrari is with the EndoCAS Center, Department of Translational Research and New Technologies in Medicine and Surgery, University of
Pisa and also with the Vascular Surgery Unit, Cisanello University Hospital AOUP.
V.
Ferrari is with the EndoCAS Center, Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa
and also with the Information Engineering Department, University of Pisa.
This work was supported by the Surgical Training in identification and isolation of tubular structures with hybrid Augmented Reality Simulation
(SThARS) project (project code GR-2011-02347124) under the grant Young Researcher (under 40 years)/Giovani Ricercatori 2011–2012, by the
Italian Minister of Health with the cofounding of Tuscany Region, the Virtual and Augu- mented Reality Support for Transcatheter Valve
Implantation, by using Cardiovascular MRI (VIVIR) project (project code PE-2013-02357974) under the grant Ricerca Finalizzata, and by the Italian
Minister of Health with the cofounding of Tuscany Region and the HORIZON 2020 Project
VOSTARS, Project ID: 731974. Call: ICT-29-2016 –
Photonics KET 2016.
(Corresponding author: Rosanna M. Viglialoro.)
Index Terms—Surgical simulator, laparoscopic simulation, augmented reality, hybrid simulators, physical anatomical
model, cholecystectomy training.
Abstract
Hybrid surgical simulators based on augmented reality (AR) solutions benefit from the advantages
of both the box trainers and the virtual reality simulators. This paper reports on the results of a long
development stage of a hybrid simulator for laparoscopic cholecystectomy that integrates real and
the virtual components. We first outline the specifications of the AR simulator and then we explain
the strategy adopted for implementing it based on a careful selection of its simulated anatomical
components, and characterized by a real-time tracking of both a target anatomy and of the
laparoscope. The former is tracked by means of an electromagnetic field generator, while the latter
requires an additional camera for video tracking. The new system was evaluated in terms of AR
visualization accuracy, realism, and hardware robustness. Obtained results show that the accuracy
of AR visualization is adequate for training purposes. The qualitative evaluation confirms the
robust- ness and the realism of the simulator. In conclusion, the proposed AR simulator satisfies all
the initial specifications in terms of anatomical appearance, modularity, reusability, minimization of
spare parts cost, and ability to record surgical errors and to track in real-time the Calot’s triangle
and the laparoscope. Thus, the proposed system could be an effective training tool for learning the
task of identification and isolation of Calot’s triangle in laparoscopic cholecystectomy. Moreover,
the presented strategy could be applied to simulate other surgical procedures involving the task of
identification and isolation of generic tubular structures, such as blood vessels, biliary tree, and
nerves, which are not directly visible.

I. Introduction
Several medical simulators exist today on the market mainly due to the increasing demand for
minimally invasive surgical (MIS) procedures and the increasing concern on patient safety [1]. Only
2 years ago the front cover of IEEE Pulse was dedicated to medical simulation with the subtitle
“New Tech Reconfigures the Training Landscapes” [2].
Current MIS simulators are mainly implemented with two opposed approaches: virtual reality (VR)
simulators, and physical simulators; each category comes with its own advantages and
disadvantages [3].
VR simulators allow the repetitive execution of basic tasks (cutting, grasping, suturing etc.) and of
the whole surgical procedures (such as laparoscopic and endoscopic procedures e.g.,
cholecystectomy), and they provide precise metrics for the evaluation of the trainee performance
[4]. Such simulators range from serious gaming applications which are becoming more widespread
to complex simulation platform [5], [6]. For example, Touch Surgery (TS) (Kinosis Limited,
London, UK), is a serious gaming cost-free application for cognitive task simulation and rehearsal
of key surgical steps [7].
On the other hand, an example of complex simulation platform is represented by LapMentor
(Simbionix, USA). This is a trainer for laparoscopic surgery available in two versions with and
without force feedback. Both versions have many modules including fundamental laparoscopic
skills and full laparoscopic procedures (e.g., cholecystectomy, etc.). Numerous studies have
demonstrated its validity. However, the complex simulation plat- forms require ongoing technical
support and system upgrades and they have the costs very high making it prohibitively expensive
for many institutions [8]–[10].
Overall, a limitation of VR simulators is the unrealistic simulation of the visual and haptic
sensations between virtual objects and instruments [8].
The physical simulators are used for the acquisition of fundamental skills, such as hand-eye
coordination, perception of depth of field, and manual skills needed for performing specific surgical
tasks comprising the use of real instruments (e.g., suturing, dissection etc.) [11]. Such simulators
range from simple box trainers to human torso models with simple objects such as pegs and
inanimate models of human organs.
One important advantage of the physical simulators is the natural haptic feedback during the task
execution. Besides, their costs are relatively low (a few thousand dollars) [9]. However, major
disadvantages of this second approach are: the lack of an automatic evaluation of the user’s
performance, its non- reusability in case of destructive tasks, and the difficulty in providing highly
detailed and anthropomorphic replicas of the organs.
A relatively innovative approach is the use of AR technology to implement hybrid simulators [12]–
[16]. According to the physical-virtual simulation spectrum proposed by Samsun Lampotang et al.
[17], AR simulation is a form of mixed simulation which combines physical simulation (such as
laparoscopic equipment or/and mannequins) and VR simulation within a unique simulation
environment.
AR simulators retain the natural haptic feedback of the physical simulation and the performance
evaluation tools typical of VR simulation [12].
Barsom et al. [18] presented a systematic review on AR simulators for medical training. The
authors divided the AR simulators into three categories on the bases of training purposes:
neurosurgical procedures, echocardiography and laparoscopic surgery.
We focused our attention on the last category. Specifically, existing AR surgical simulators for MIS
training are very few both at commercial and research level. The only commercial AR surgical
simulator is the ProMIS (Haptica, Dublin, Ireland). Today its production is temporarily stopped
because the system was acquired by CAE Healthcare. The system offers the simulation of basic
skills and some laparoscopic procedures (e.g., appendectomy, colectomy, cholecystectomy). The
system com- bines a laparoscopic mannequin connected to a laptop and a virtual environment. It

uses real surgical instruments which are tracked during the tasks to provide an accurate and
objective assessment of the user’s performance. Different physical models such as suturing pads,
can be inserted into the mannequin.
However, the included physical models are neither highly detailed nor anthropomorphic. In
addition, the simulator does not allow either to update virtual anatomy following deformations
impressed on physical anatomical models or to add simulated physiological functionalities. Indeed,
virtual information is applied to complete the surgical scene with organs not included in the
synthetic environment, and to provide visual instructions and guidance information: for example, it
indicates where to cut a tissue and it shows the “virtual jet” of blood in case of tissue damages [8].
To date, ProMIS is the most validated surgical simulator and, as already said, the only commercial
system featuring AR functionalities [12], [13], [19].
Concerning the non-commercial AR laparoscopic simulators, Lahanas et al. have described an AR
simulator for MIS basic skills (e.g., navigation, peg transfer and clipping). The system allows the
real-time tracking of the laparoscopic instruments posed inside a box trainer and the interaction with
various virtual elements rendered on the screen. The authors provide an evidence for face and
construct validity [8], [20].
However, existing AR laparoscopic simulators have a limited AR experience and they allow only
the execution of simple tasks. In addition, they are often based on physical models that are not
highly detailed, non-anthropomorphic.
To the best of our knowledge, we believe that so far, the real potentialities offered by AR
technologies have not been explored yet in the context of surgical simulation.
Accordingly, we explored the potentialities of AR in surgical simulation for the training of
identification and isolation of deformable tubular structures in surgery [21], [22].
In [21], we described an AR proof-of-concept solution that allows the AR visualization of hidden
tubular structures sensorized with miniature electromagnetic (EM) sensor coils. In the same paper,
we also motivated why the use of EM tracking is the best technology available nowadays.
Furthermore, in [22], we applied the proposed AR solution to visualize and to recognize different
configurations of the Calot’s triangle in simulated cholecystectomy procedures. In all these works
the augmented scene was generated by combining the image captured by a fixed camera and the
information from the EM tracking.
In this paper, we report on the lessons learned during the development of an AR advanced
simulation platform for laparoscopic cholecystectomy (LC), and we show the results of its
evaluation. LC is the standard of care for gallbladder removal and the most commonly performed
laparoscopic surgical procedure [23].
However, a serious and devastating technical complication of the LC is bile duct injury (BDI) [24].
There are mainly two factors increasing the BDI risk: the misinterpretation of the anatomy,
mentioned by 92.9% of surgeons as the primary factor, and the lack of experience in LC, mentioned
by 70.9% of surgeons as a contributing factor [25]. Indeed, although laparoscopic surgery (LS) is
beneficial to the patient (small incisions, short hospital stays, minimal postoperative pain, etc.), it is
technically more demanding and stressful than the traditional surgery, and it requires greater
concentration [26].
General complications of LS are often related to the commonly known drawbacks of the
laparoscopic environment: the two-dimensional images provided by the laparoscope do not allow a
clear-cut vision as in the open intervention, therefore the surgeon significantly loses the perception
of depth; the instruments are difficult to manipulate due to the fulcrum effect; the use of minimally
invasive surgical tools causes the loss of tactile sensation [27]–[29]. To prevent avoidable surgical
complications and to fully master the procedure, the surgeon must acquire the specific technical
skills which can be achieved through appropriate training in LS. For this purpose, the simulation
offers a safe and practical way to gain and to assess skills, through repeated practice, outside the
operating theatre [30].

As already mentioned, in our previous studies [21], [22], [31], we presented a proof-of-concept of
AR simulation platform for LC to practice identification and isolation of the Calot’s triangle. Based
on our previous works, this paper presents the final design specifications of the AR simulator taking
into account the iterative feedback obtained by surgeons who have tested our previous
demonstrators. In detail, we described an innovative simulation strategy which combines highly
detailed physical models and virtual reality information in a surgical scene. The strategy deeply
integrates both visual AR and acoustic functionalities, improving the state of the art. In comparison
to our proof-of -concept system, the current system includes the localization of laparoscope.
Specifically, we present and implement a solution for calibration that involves an additional camera
for video tracking. The localization of laparoscope allows its free movement maintaining the
geometric coherence of the AR scene. The quantitative tests are aimed to evaluate the AR
visualization accuracy. In addition, not only the usefulness of AR visualization but also the
experimental results for the evaluation of the complete system in terms of both robustness and
realism of the physical components have been reported for the first time, in this paper.
Finally, this manuscript provides manufacturing details for the implementation of the simulator.
The paper is organized as follows:
• An overview about design specifications and a detailed description of the physical
components of the simulator.
• The methodologies for AR implementation and laparoscope tracking and the results of
the experimental tests. The tests are aimed to evaluate the AR visualization accuracy
and qualitatively verify the simulator robustness and realism.
• A thorough discussion about all the issues addressed during the development of the
simulator and possible future improvements and further fields of applications.
II. Material and Methods
A. Design Specifications
The basic design specifications for our hybrid LC simulator are: realistic anatomical appearance,
modularity, reusability, minimization of spare parts cost, AR visualization, ability to signal surgical
errors and to track in real time both Calot’ s triangle structures and laparoscope. Here follows a
detailed description of the specifications:
1) Realistic anatomical appearance: The simulated anatomy is chosen taking into account all
the anatomical structures that could be either seen or touched during the execution of the
procedure.
The anatomical replicas should match morphology, topology, color, texture, density and they
should mimic the mechanical behavior of the real structures as far as possible. Therefore, our
simulator includes the following replicas: liver, gallbladder, biliary tree, arterial tree, pancreas,
abdominal aorta, esophagus- stomach-duodenum and connective tissue.
2) Modularity: Due to the morphological and topological variations that occur naturally in
human hepatobiliary anatomy, the prototype is designed to be modular, so that anatomical parts
such as connective tissue, biliary tree (BT) and arterial tree (AT) can be easily and separately re-
placed. In particular, BT and AT, which are sensorized and thus expensive, are designed to be
reusable. In our sim- ulator, different anatomical variations are predisposed, and an easy
connection/disconnection coupling is imple- mented to allow any easy substitutions on demand, for
training sessions with different complexity. The connec- tive tissue, fabricated with a low-cost
material, is instead disposable.
3) Reusability and minimization of spare parts cost: Reusability and minimization of spare
parts cost are closely linked. All the anatomical structures, except from the connective tissue which
must be dissected, are to be fabricated using materials that are reusable and extremely durable over

time. For this reason, silicone and nitinol tubes are used in order that the overall training costs are
reduced.
4) AR visualization: The simulator offers an AR visualiza- tion of the hidden tubular structures
(AT and BT). AR visualization has twofold function: it provides the trainee information about the
implemented anatomical variations and it provides a way to verify the correct isolation of the target
structures after the trial.
5) Signaling surgical errors: The prototype is designed to enable acoustic signaling of
potential damages to the BT and/or AT walls. In this way, the simulator can directly evidence a lack
of surgical skills and provide a way to evaluate the performance.
6) EM tracking of the Calot’s triangle structures: The proto- type is designed to enable the EM
tracking of the Calot’s triangle structures.
7) Localization of the laparoscope: The prototype is de- signed to enable the real-time tracking
of the laparoscope. In this way, it is possible to freely move the laparoscope maintaining the
geometric coherence of the AR scene. This aspect is fundamental during laparoscopic proce- dures.
To this end, the strategy requires an additional camera for optical detection and tracking of a
structured marker.
In the following sections, we outline the steps that led to the development of the final simulator,
from the Computer Aided Design (CAD) project to its fabrication, and then we describe a
preliminary evaluation of the simulator. Sections are:
B) CAD Project of the real simulator components
C) Fabrication of the real simulator components
D) Overview of AR simulator
E) Evaluation of AR simulator
Fig. 1. CAD project of the simulator real components: a) 3D model of whole simulator; b-i) 3D model of each component of
simulator: b) Esophagus-Stomach-Duodenum; c) Connective tissue, d) Gallbladder;
e) Biliary tree; f) Arterial tree; g) Aorta; h) Liver; i) Pancreas
B. CAD Project of the Real Simulator Components
The first step involves the CAD project of the whole real simulator components by means of the
PTC Creo Parametric
2.0 software. The CAD project integrates both 3D anatomical models and 3D models of the
required electronic accessories (Fig. 1).
C. Fabrication of the Real Simulator Components
1) Simulator Components: Based on the adopted fabrica- tion strategy [21], [22], [31], [32],
the anatomical real compo- nents of the simulator can be divided in two categories:

Frequently asked questions

Q1. What have the authors contributed in "Augmented reality to improve surgical simulation: lessons learned towards the design of a hybrid laparoscopic simulator for cholecystectomy" ?

This paper reports on the results of a long development stage of a hybrid simulator for laparoscopic cholecystectomy that integrates real and the virtual components. The authors first outline the specifications of the AR simulator and then they explain the strategy adopted for implementing it based on a careful selection of its simulated anatomical components, and characterized by a real-time tracking of both a target anatomy and of the laparoscope. 

Q2. What are the future works mentioned in the paper "Augmented reality to improve surgical simulation: lessons learned towards the design of a hybrid laparoscopic simulator for cholecystectomy" ?

Further studies will focus on demonstration of the effective- ness, of the validity and of the appropriateness of the simulator as a training tool for novices. 

Q3. What is the process of constructing the patient specific models?

The strategy used for the patient specific models’ fabrication consists of four 4 main steps: extraction of the 3D models of the target organs starting from CT images; molds designing in the 3D CAD software; molds manufacturing with a 3D printer (Dimension Elite 3D Printer); casting of the chosen materials into the molds. 

Q4. What were the fragile parts of the simulator?

In particular, the most fragile parts of the entire simulator were the electrical connections between the thin sen- sor wires (about 0.018 mm) and the connectors which are the interface between the Calot’ s triangle and the rest of the system. 

Q5. What are the main factors that affect the accuracy of the EM field generator?

The obtained accuracy is mainly affected by four sources of error: the inherent accuracy of EM tracking paired with the field distortions arising from the environment; the difficulties in positioning the EM sensors at the tubular structures centerline [17]; the interpolation error when drawing each virtual tract; and the errors accumulated during the system calibration. 

Q6. What is the purpose of the presented strategy?

The presented strategy could be applied to simulate surgical laparoscopic procedures involving the task of identification and isolation of other generic tubular structures, such as blood vessels, and nerves, which are not visible. 

Q7. What is the purpose of the calibration of the laparoscope and the USB camera?

An accurate calibration of the intrinsic (projective) and extrinsic (pose) parameters of the laparoscope and of the USB camera is essential for accurate AR visualization. 

Q8. What is the way to create a session with different complexity?

This gives the opportunity to create session’s training with different complexity and it will allow the trainee to acquire both the dexterity necessary for good practice and the decision- making skills. 

Q9. What is the strategy used for the fabrication of the con- nective tissue?

the strategy adopted for the fabrication of the con- nective tissue, which has to be dissected during the simulation and then replaced for each trail, involves the use of a degradable material. 

Q10. What is the simplest way to change the horizon of the laparoscope?

Since in such laparoscopes it is possible to rotate the optics with respect to the camera head in order to change the image “horizon”, the authors 3D printed an ABS block to block this degree of freedom. 

Q11. What is the urgent improvement for the simulator?

the most urgent improvement will be the use of shielded and isolated EM sensor coils able to reduce electrical interference and to offer more mechanical robustness. 

Q12. What is the process for the simulation of the Calot’s triangle?

The manufacturing process takes into account the need of: . sensorizing the Calot’s triangle for the implementation of the AR functionalities; . using conductive materials for the detection of electric contact between BT and/or AT and the surgical instru- ments by means of sound warnings. 

Q13. What is the use of a conductive material for the simulation of the Calot’s?

As already described in [21], commercially available 5 de- grees of freedom (DOF) EM sensor coils (0.50 mm diameter, 8 mm length) are used to implement an AR solution allowing the real-time visualization of the Calot’s triangle. 

Q14. What is the way to visualize the Calot’s triangle?

After the early stages of the learning curve, the AR visual- ization of the Calot’s triangle should be turned on only in case of an occurring surgical damage to the sensorized structure.