Showing papers presented at "Computer Assisted Radiology and Surgery in 2015"

Multicenter study on the use of patient-specific CAD/CAM reconstruction plates for mandibular reconstruction

Abstract: For the new generation of mandibular reconstruction, patient-specific mandible reconstruction plates (PSMPs) have been developed which are milled from titanium after preoperative computer planning using CAD/CAM procedures. Resection margins and plate position are determined by surgical guides. In addition, length and shape of the plate and the number and angulation of the screw holes can be planned. 30 patients received such PSMP. Indication ranged from stabilization osteosynthesis, single alloplastic stand-alone reconstruction to microvascular reconstructions. Time for planning, fit of surgical guides and plates, pre-/postoperative occlusion, radiological position of the temporo mandibular joint and complications were recorded. The median time for online planning was 35 min. The results concerning fit and handling of the PSMP and the surgical guides were mainly very positive. In six cases, the plan had to be adapted to the intraoperative clinical needs. The postoperative position of the condyles in the temporo mandibular fossae was regular in 28 cases. The evaluation of the occlusion was not representative due to not clearly identifiable occlusion in 2/3 of the cases. Nevertheless, complications like postoperative extraoral plate exposure, infection, graft and flap necrosis or difficulties to position the guides or the plate during surgery occurred. Mandibular reconstruction with PSMP offers a broad range of opportunities and benefits compared with standard procedures and can be recommended for all kind of mandibular reconstructions. It is not yet foreseeable whether PSMP will in future become routine clinical practice for mandibular reconstruction or will be confined to selected isolated cases.

A knowledge-based algorithm for automatic detection of cephalometric landmarks on CBCT images

Abstract: Cone-beam computed tomography (CBCT) is now an established component for 3D evaluation and treatment planning of patients with severe malocclusion and craniofacial deformities. Precision landmark plotting on 3D images for cephalometric analysis requires considerable effort and time, notwithstanding the experience of landmark plotting, which raises a need to automate the process of 3D landmark plotting. Therefore, knowledge-based algorithm for automatic detection of landmarks on 3D CBCT images has been developed and tested. A knowledge-based algorithm was developed in the MATLAB programming environment to detect 20 cephalometric landmarks. For the automatic detection, landmarks that are physically adjacent to each other were clustered into groups and were extracted through a volume of interest (VOI). Relevant contours were detected in the VOI and landmarks were detected using corresponding mathematical entities. The standard data for validation were generated using manual marking carried out by three orthodontists on a dataset of 30 CBCT images as a reference. Inter-observer ICC for manual landmark identification was found to be excellent ( $$>$$ 0.9) amongst three observers. Euclidean distances between the coordinates of manual identification and automatic detection through the proposed algorithm of each landmark were calculated. The overall mean error for the proposed method was 2.01 mm with a standard deviation of 1.23 mm for all the 20 landmarks. The overall landmark detection accuracy was recorded at 64.67, 82.67 and 90.33 % within 2-, 3- and 4-mm error range of manual marking, respectively. The proposed knowledge-based algorithm for automatic detection of landmarks on 3D images was able to achieve relatively accurate results than the currently available algorithm.

Augmented reality in neurovascular surgery: feasibility and first uses in the operating room

Abstract: The aim of this report is to present a prototype augmented reality (AR) intra-operative brain imaging system. We present our experience of using this new neuronavigation system in neurovascular surgery and discuss the feasibility of this technology for aneurysms, arteriovenous malformations (AVMs), and arteriovenous fistulae (AVFs). We developed an augmented reality system that uses an external camera to capture the live view of the patient on the operating room table and to merge this view with pre-operative volume-rendered vessels. We have extensively tested the system in the laboratory and have used the system in four surgical cases: one aneurysm, two AVMs and one AVF case. The developed AR neuronavigation system allows for precise patient-to-image registration and calibration of the camera, resulting in a well-aligned augmented reality view. Initial results suggest that augmented reality is useful for tailoring craniotomies, localizing vessels of interest, and planning resection corridors. Augmented reality is a promising technology for neurovascular surgery. However, for more complex anomalies such as AVMs and AVFs, better visualization techniques that allow one to distinguish between arteries and veins and determine the absolute depth of a vessel of interest are needed.

NiftySim: A GPU-based nonlinear finite element package for simulation of soft tissue biomechanics

Abstract: NiftySim, an open-source finite element toolkit, has been designed to allow incorporation of high-performance soft tissue simulation capabilities into biomedical applications. The toolkit provides the option of execution on fast graphics processing unit (GPU) hardware, numerous constitutive models and solid-element options, membrane and shell elements, and contact modelling facilities, in a simple to use library. The toolkit is founded on the total Lagrangian explicit dynamics (TLEDs) algorithm, which has been shown to be efficient and accurate for simulation of soft tissues. The base code is written in C $$++$$ , and GPU execution is achieved using the nVidia CUDA framework. In most cases, interaction with the underlying solvers can be achieved through a single Simulator class, which may be embedded directly in third-party applications such as, surgical guidance systems. Advanced capabilities such as contact modelling and nonlinear constitutive models are also provided, as are more experimental technologies like reduced order modelling. A consistent description of the underlying solution algorithm, its implementation with a focus on GPU execution, and examples of the toolkit’s usage in biomedical applications are provided. Efficient mapping of the TLED algorithm to parallel hardware results in very high computational performance, far exceeding that available in commercial packages. The NiftySim toolkit provides high-performance soft tissue simulation capabilities using GPU technology for biomechanical simulation research applications in medical image computing, surgical simulation, and surgical guidance applications.

Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty

Abstract: Previous work on surgical skill assessment using intraoperative tool motion has focused on highly structured surgical tasks such as cholecystectomy and used generic motion metrics such as time and number of movements. Other statistical methods such as hidden Markov models (HMM) and descriptive curve coding (DCC) have been successfully used to assess skill in structured activities on bench-top tasks. Methods to assess skill and provide effective feedback to trainees for unstructured surgical tasks in the operating room, such as tissue dissection in septoplasty, have yet to be developed. We proposed a method that provides a descriptive structure for septoplasty by automatically segmenting it into higher-level meaningful activities called strokes. These activities characterize the surgeon’s tool motion pattern. We constructed a spatial graph from the sequence of strokes in each procedure and used its properties to train a classifier to distinguish between expert and novice surgeons. We compared the results from our method with those from HMM, DCC, and generic metric-based approaches. We showed that our method—with an average accuracy of 91 %—performs better or equal than these state-of-the-art methods, while simultaneously providing surgeons with an intuitive understanding of the procedure. In this study, we developed and evaluated an automated approach to objectively assess surgical skill during unstructured task of tissue dissection in nasal septoplasty.

Multi-contrast unbiased MRI atlas of a Parkinson’s disease population

Abstract: Parkinson’s disease (PD) is the second leading neurodegenerative disease after Alzheimer’s disease. In PD research and its surgical treatment, such as deep brain stimulation (DBS), anatomical structural identification and references for spatial normalization are essential, and human brain atlases/templates are proven highly instrumental. However, two shortcomings affect current templates used for PD. First, many templates are derived from a single healthy subject that is not sufficiently representative of the PD-population anatomy. This may result in suboptimal surgical plans for DBS surgery and biased analysis for morphological studies. Second, commonly used mono-contrast templates lack sufficient image contrast for some subcortical structures (i.e., subthalamic nucleus) and biochemical information (i.e., iron content), a valuable feature in current PD research. We employed a novel T1–T2* fusion MRI that visualizes both cortical and subcortical structures to drive groupwise registration to create co-registered multi-contrast (T1w, T2*w, T1–T2* fusion, phase, and an R2* map) unbiased templates from 15 PD patients, and a high-resolution histology-derived 3D atlas is co-registered. For validation, these templates are compared against the Colin27 template for landmark registration and midbrain nuclei segmentation. While the T1w, T2*w, and T1–T2* fusion templates provide excellent anatomical details for both cortical and subcortical structures, the phase and R2* map contain the biochemical features. By one-way ANOVA tests, our templates significantly ( $$p<0.05$$ ) outperform the Colin27 template in the registration-based tasks. The proposed unbiased templates are more representative of the population of interest and can benefit both the surgical planning and research of PD.

Neurosurgical virtual reality simulation metrics to assess psychomotor skills during brain tumor resection

Abstract: Virtual reality simulator technology together with novel metrics could advance our understanding of expert neurosurgical performance and modify and improve resident training and assessment. This pilot study introduces innovative metrics that can be measured by the state-of-the-art simulator to assess performance. Such metrics cannot be measured in an operating room and have not been used previously to assess performance. Three sets of performance metrics were assessed utilizing the NeuroTouch platform in six scenarios with simulated brain tumors having different visual and tactile characteristics. Tier 1 metrics included percentage of brain tumor resected and volume of simulated “normal” brain tissue removed. Tier 2 metrics included instrument tip path length, time taken to resect the brain tumor, pedal activation frequency, and sum of applied forces. Tier 3 metrics included sum of forces applied to different tumor regions and the force bandwidth derived from the force histogram. The results outlined are from a novice resident in the second year of training and an expert neurosurgeon. The three tiers of metrics obtained from the NeuroTouch simulator do encompass the wide variability of technical performance observed during novice/expert resections of simulated brain tumors and can be employed to quantify the safety, quality, and efficiency of technical performance during simulated brain tumor resection. Tier 3 metrics derived from force pyramids and force histograms may be particularly useful in assessing simulated brain tumor resections. Our pilot study demonstrates that the safety, quality, and efficiency of novice and expert operators can be measured using metrics derived from the NeuroTouch platform, helping to understand how specific operator performance is dependent on both psychomotor ability and cognitive input during multiple virtual reality brain tumor resections.

Multimodal image-guided prostate fusion biopsy based on automatic deformable registration

Abstract: Transrectal ultrasound (TRUS)-guided random prostate biopsy is, in spite of its low sensitivity, the gold standard for the diagnosis of prostate cancer. The recent advent of PET imaging using a novel dedicated radiotracer, $$^{68}\hbox {Ga}$$ -labeled prostate-specific membrane antigen (PSMA), combined with MRI provides improved pre-interventional identification of suspicious areas. This work proposes a multimodal fusion image-guided biopsy framework that combines PET-MRI images with TRUS, using automatic segmentation and registration, and offering real-time guidance. The prostate TRUS images are automatically segmented with a Hough transform-based random forest approach. The registration is based on the Coherent Point Drift algorithm to align surfaces elastically and to propagate the deformation field calculated from thin-plate splines to the whole gland. The method, which has minimal requirements and temporal overhead in the existing clinical workflow, is evaluated in terms of surface distance and landmark registration error with respect to the clinical ground truth. Evaluations on agar–gelatin phantoms and clinical data of 13 patients confirm the validity of this approach. The system is able to successfully map suspicious regions from PET/MRI to the interventional TRUS image.

Mobile markerless augmented reality and its application in forensic medicine

Abstract: During autopsy, forensic pathologists today mostly rely on visible indication, tactile perception and experience to determine the cause of death. Although computed tomography (CT) data is often available for the bodies under examination, these data are rarely used due to the lack of radiological workstations in the pathological suite. The data may prevent the forensic pathologist from damaging evidence by allowing him to associate, for example, external wounds to internal injuries. To facilitate this, we propose a new multimodal approach for intuitive visualization of forensic data and evaluate its feasibility. A range camera is mounted on a tablet computer and positioned in a way such that the camera simultaneously captures depth and color information of the body. A server estimates the camera pose based on surface registration of CT and depth data to allow for augmented reality visualization of the internal anatomy directly on the tablet. Additionally, projection of color information onto the CT surface is implemented. We validated the system in a postmortem pilot study using fiducials attached to the skin for quantification of a mean target registration error of $$4.4 \pm 1.3$$ mm. The system is mobile, markerless, intuitive and real-time capable with sufficient accuracy. It can support the forensic pathologist during autopsy with augmented reality and textured surfaces. Furthermore, the system enables multimodal documentation for presentation in court. Despite its preliminary prototype status, it has high potential due to its low price and simplicity.

LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition

Abstract: The rise of intraoperative information threatens to outpace our abilities to process it. Context-aware systems, filtering information to automatically adapt to the current needs of the surgeon, are necessary to fully profit from computerized surgery. To attain context awareness, representation of medical knowledge is crucial. However, most existing systems do not represent knowledge in a reusable way, hindering also reuse of data. Our purpose is therefore to make our computational models of medical knowledge sharable, extensible and interoperational with established knowledge representations in the form of the LapOntoSPM ontology. To show its usefulness, we apply it to situation interpretation, i.e., the recognition of surgical phases based on surgical activities. Considering best practices in ontology engineering and building on our ontology for laparoscopy, we formalized the workflow of laparoscopic adrenalectomies, cholecystectomies and pancreatic resections in the framework of OntoSPM, a new standard for surgical process models. Furthermore, we provide a rule-based situation interpretation algorithm based on SQWRL to recognize surgical phases using the ontology. The system was evaluated on ground-truth data from 19 manually annotated surgeries. The aim was to show that the phase recognition capabilities are equal to a specialized solution. The recognition rates of the new system were equal to the specialized one. However, the time needed to interpret a situation rose from 0.5 to 1.8 s on average which is still viable for practical application. We successfully integrated medical knowledge for laparoscopic surgeries into OntoSPM, facilitating knowledge and data sharing. This is especially important for reproducibility of results and unbiased comparison of recognition algorithms. The associated recognition algorithm was adapted to the new representation without any loss of classification power. The work is an important step to standardized knowledge and data representation in the field on context awareness and thus toward unified benchmark data sets.

Electromagnetic tracking in surgical and interventional environments: usability study

Abstract: Electromagnetic (EM) tracking of instruments within a clinical setting is notorious for fluctuating measurement performance. Position location measurement uncertainty of an EM system was characterized in various environments, including control, clinical, cone beam computed tomography (CBCT), and CT scanner environments. Static and dynamic effects of CBCT and CT scanning on EM tracking were evaluated. Two guidance devices were designed to solely translate or rotate the sensor in a non-interfering fit to decouple pose-dependent tracking uncertainties. These devices were mounted on a base to allow consistent and repeatable tests when changing environments. Using this method, position and orientation measurement accuracies, precision, and 95 % confidence intervals were assessed. The tracking performance varied significantly as a function of the environment—especially within the CBCT and CT scanners—and sensor pose. In fact, at a fixed sensor position in the clinical environment, the measurement error varied from 0.2 to 2.2 mm depending on sensor orientations. Improved accuracies were observed along the vertical axis of the field generator. Calibration of the measurements improved tracking performance in the CT environment by 50–85 %. EM tracking can provide effective assistance to surgeons or interventional radiologists during procedures performed in a clinical or CBCT environment. Applications in the CT scanner demand precalibration to provide acceptable performance.

PyDBS: an automated image processing workflow for deep brain stimulation surgery

Abstract: Deep brain stimulation (DBS) is a surgical procedure for treating motor-related neurological disorders. DBS clinical efficacy hinges on precise surgical planning and accurate electrode placement, which in turn call upon several image processing and visualization tasks, such as image registration, image segmentation, image fusion, and 3D visualization. These tasks are often performed by a heterogeneous set of software tools, which adopt differing formats and geometrical conventions and require patient-specific parameterization or interactive tuning. To overcome these issues, we introduce in this article PyDBS, a fully integrated and automated image processing workflow for DBS surgery. PyDBS consists of three image processing pipelines and three visualization modules assisting clinicians through the entire DBS surgical workflow, from the preoperative planning of electrode trajectories to the postoperative assessment of electrode placement. The system’s robustness, speed, and accuracy were assessed by means of a retrospective validation, based on 92 clinical cases. The complete PyDBS workflow achieved satisfactory results in 92 % of tested cases, with a median processing time of 28 min per patient. The results obtained are compatible with the adoption of PyDBS in clinical practice.

Mandible reconstruction using patient-specific pre-bent reconstruction plates: comparison of standard and transfer key methods.

Abstract: Mandible reconstruction with reconstruction plates requires bending the plates during the operation and fixation using the “standard method” (ST-method). The ST-method is limited when a pathological process has perforated the mandibular outer cortex. A transfer key method (TK-method) was developed where plates are pre-bent using a patient-specific mandible model and positioned on the mandible with the help of transfer keys. The ST-method and TK-method were compared in a clinical trial. Mandibular reconstruction was performed on 42 patients in this study: 22 were performed using the TK-method and 20 using the ST-method. Pre- and postoperative CT scans were evaluated by measuring the distances between six corresponding landmarks on the mandibular condyles and rami. The difference between pre- and postoperative distances was used to evaluate reconstruction accuracy. The median deviation of the unsigned/ absolute values of all six distances was 1.07 mm for the TK-method and 1.67 mm for the ST-method. The TK-method showed significantly better results. For the signed values, the median deviation of the six distances was $$-$$ 0.6 mm for the TK-method and $$-$$ 1.47 mm for the ST-method, indicating that the mandibles became narrower with both methods. This width difference was not statistically significant. The TK-method was more accurate than the ST-method in a clinical trial. The TK-method was effective and accurate for mandible reconstruction using pre-bent fixation plates.

A system for context-aware intraoperative augmented reality in dental implant surgery

Abstract: Large volumes of information in the OR are ignored by surgeons when the amount outpaces human mental processing abilities. We developed an augmented reality (AR) system for dental implant surgery that acts as an automatic information filter, selectively displaying only relevant information. The purpose is to reduce information overflow and offer intuitive image guidance. The system was evaluated in a pig cadaver experiment. Information filtering is implemented via rule-based situation interpretation with description logics. The interpretation is based on intraoperative distances measurement between anatomical structures and the dental drill with optical tracking. For AR, a head-mounted display is used, which was calibrated with a novel method based on SPAAM. To adapt to surgeon specific preferences, we offer two alternative display formats: one with static and another with contact analog AR. The system made the surgery easier and showed ergonomical benefits, as assessed by a questionnaire. All relevant phases were recognized reliably. The new calibration showed significant improvements, while the deviation of the realized implants was $$<$$ 2.5 mm. The system allowed the surgeon to fully concentrate on the surgery itself. It offered greater flexibility since the surgeon received all relevant information, but was free to deviate from it. Accuracy of the realized implants remains an open issue and part of future work.

Interactive multi-criteria planning for radiofrequency ablation

Abstract: Image-guided radiofrequency ablation (RFA) is a broadly used minimally invasive method for the thermal destruction of focal liver malignancies using needle-shaped instruments. The established planning workflow is based on examination of 2D slices and manual definition of the access path. During that process, multiple criteria for all possible trajectories have to be taken into account. Hence, it demands considerable experience and constitutes a significant mental task. An access path determination method based on image processing and numerical optimization is proposed. Fast GPU-based simulation approximation is utilized to incorporate the heat distribution including realistic cooling effects from nearby blood vessels. A user interface for intuitive exploration of the optimization results is introduced. The proposed methods are integrated into a clinical software assistant. To evaluate the suitability of the interactive optimization approach for the identification of meaningful therapy strategies, a retrospective study has been carried out. The system is able to propose clinically relevant trajectories to the target by incorporating multiple criteria. A novel method for planning of image-guided radiofrequency ablation by means of interactive access path determination based on optimization is presented. A first retrospective study indicates that the method is suited to improve the classical planning of RFA.

Haptic computer-assisted patient-specific preoperative planning for orthopedic fractures surgery

Abstract: Purpose The aim of orthopedic trauma surgery is to restore the anatomy and function of displaced bone fragments to support osteosynthesis. For complex cases, including pelvic bone and multi-fragment femoral neck and distal radius fractures, preoperative planning with a CT scan is indicated. The planning consists of (1) fracture reduction—determining the locations and anatomical sites of origin of the fractured bone fragments and (2) fracture fixation—selecting and placing fixation screws and plates. The current bone fragment manipulation, hardware selection, and positioning processes based on 2D slices and a computer mouse are time-consuming and require a technician.

The NifTK software platform for image-guided interventions: platform overview and NiftyLink messaging.

Abstract: Purpose To perform research in image-guided interventions, researchers need a wide variety of software components, and assembling these components into a flexible and reliable system can be a challenging task. In this paper, the NifTK software platform is presented. A key focus has been high-performance streaming of stereo laparoscopic video data, ultrasound data and tracking data simultaneously.

Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms

Abstract: Context-aware systems for the operating room (OR) provide the possibility to significantly improve surgical workflow through various applications such as efficient OR scheduling, context-sensitive user interfaces, and automatic transcription of medical procedures. Being an essential element of such a system, surgical action recognition is thus an important research area. In this paper, we tackle the problem of classifying surgical actions from video clips that capture the activities taking place in the OR. We acquire recordings using a multi-view RGBD camera system mounted on the ceiling of a hybrid OR dedicated to X-ray-based procedures and annotate clips of the recordings with the corresponding actions. To recognize the surgical actions from the video clips, we use a classification pipeline based on the bag-of-words (BoW) approach. We propose a novel feature encoding method that extends the classical BoW approach. Instead of using the typical rigid grid layout to divide the space of the feature locations, we propose to learn the layout from the actual 4D spatio-temporal locations of the visual features. This results in a data-driven and non-rigid layout which retains more spatio-temporal information compared to the rigid counterpart. We classify multi-view video clips from a new dataset generated from 11-day recordings of real operations. This dataset is composed of 1734 video clips of 15 actions. These include generic actions (e.g., moving patient to the OR bed) and actions specific to the vertebroplasty procedure (e.g., hammering). The experiments show that the proposed non-rigid feature encoding method performs better than the rigid encoding one. The classifier’s accuracy is increased by over 4 %, from 81.08 to 85.53 %. The combination of both intensity and depth information from the RGBD data provides more discriminative power in carrying out the surgical action recognition task as compared to using either one of them alone. Furthermore, the proposed non-rigid spatio-temporal feature encoding scheme provides more discriminative histogram representations than the rigid counterpart. To the best of our knowledge, this is also the first work that presents action recognition results on multi-view RGBD data recorded in the OR.

Automatic phase prediction from low-level surgical activities

Abstract: Purpose: Analyzing surgical activities has received a growing interest in recent years. Several methods have been proposed to identify surgical activities and surgical phases from data acquired in operating rooms. These context-aware systems have multiple applications, including: supporting the surgical team during the intervention, improving the automatic monitoring, designing new teaching paradigms.Methods: In this paper, we use low-level recordings of the activities that are performed by a surgeon to automatically predict the current (high-level) phase of the surgery. We augment a decision tree algorithm with the ability to consider the local-context of the surgical activities and a hierarchical clustering algorithm.Results: Experiments were performed on 22 surgeries of lumbar disc herniation. We obtained an overall precision of 0.843 in detecting phases of 51,489 single activities. We also assess the robustness of the method with regard to noise.Conclusion: We show that using the local context allows us to improve the results compared with methods only considering single activity. Experiments show that the use of the local context makes our method very robust to noise and that clustering the input data first improves the predictions.

A study of crowdsourced segment-level surgical skill assessment using pairwise rankings

Abstract: Currently available methods for surgical skills assessment are either subjective or only provide global evaluations for the overall task. Such global evaluations do not inform trainees about where in the task they need to perform better. In this study, we investigated the reliability and validity of a framework to generate objective skill assessments for segments within a task, and compared assessments from our framework using crowdsourced segment ratings from surgically untrained individuals and expert surgeons against manually assigned global rating scores. Our framework includes (1) a binary classifier trained to generate preferences for pairs of task segments (i.e., given a pair of segments, specification of which one was performed better), (2) computing segment-level percentile scores based on the preferences, and (3) predicting task-level scores using the segment-level scores. We conducted a crowdsourcing user study to obtain manual preferences for segments within a suturing and knot-tying task from a crowd of surgically untrained individuals and a group of experts. We analyzed the inter-rater reliability of preferences obtained from the crowd and experts, and investigated the validity of task-level scores obtained using our framework. In addition, we compared accuracy of the crowd and expert preference classifiers, as well as the segment- and task-level scores obtained from the classifiers. We observed moderate inter-rater reliability within the crowd (Fleiss’ kappa, $$\kappa = 0.41$$ ) and experts ( $$\kappa = 0.55$$ ). For both the crowd and experts, the accuracy of an automated classifier trained using all the task segments was above par as compared to the inter-rater agreement [crowd classifier 85 % (SE 2 %), expert classifier 89 % (SE 3 %)]. We predicted the overall global rating scores (GRS) for the task with a root-mean-squared error that was lower than one standard deviation of the ground-truth GRS. We observed a high correlation between segment-level scores ( $$\rho \ge 0.86$$ ) obtained using the crowd and expert preference classifiers. The task-level scores obtained using the crowd and expert preference classifier were also highly correlated with each other ( $$\rho \ge 0.84$$ ), and statistically equivalent within a margin of two points (for a score ranging from 6 to 30). Our analyses, however, did not demonstrate statistical significance in equivalence of accuracy between the crowd and expert classifiers within a 10 % margin. Our framework implemented using crowdsourced pairwise comparisons leads to valid objective surgical skill assessment for segments within a task, and for the task overall. Crowdsourcing yields reliable pairwise comparisons of skill for segments within a task with high efficiency. Our framework may be deployed within surgical training programs for objective, automated, and standardized evaluation of technical skills.

Automatic lung segmentation method for MRI-based lung perfusion studies of patients with chronic obstructive pulmonary disease

Abstract: A novel fully automatic lung segmentation method for magnetic resonance (MR) images of patients with chronic obstructive pulmonary disease (COPD) is presented. The main goal of this work was to ease the tedious and time-consuming task of manual lung segmentation, which is required for region-based volumetric analysis of four-dimensional MR perfusion studies which goes beyond the analysis of small regions of interest. The first step in the automatic algorithm is the segmentation of the lungs in morphological MR images with higher spatial resolution than corresponding perfusion MR images. Subsequently, the segmentation mask of the lungs is transferred to the perfusion images via nonlinear registration. Finally, the masks for left and right lungs are subdivided into a user-defined number of partitions. Fourteen patients with two time points resulting in 28 perfusion data sets were available for the preliminary evaluation of the developed methods. Resulting lung segmentation masks are compared with reference segmentations from experienced chest radiologists, as well as with total lung capacity (TLC) acquired by full-body plethysmography. TLC results were available for thirteen patients. The relevance of the presented method is indicated by an evaluation, which shows high correlation between automatically generated lung masks with corresponding ground-truth estimates. The evaluation of the developed methods indicates good accuracy and shows that automatically generated lung masks differ from expert segmentations about as much as segmentations from different experts.

Generalized iterative most likely oriented-point (G-IMLOP) registration

Abstract: The need to align multiple representations of anatomy is a problem frequently encountered in clinical applications. A new algorithm for feature-based registration is presented that solves this problem by aligning both position and orientation information of the shapes being registered. The iterative most likely oriented-point (IMLOP) algorithm and its generalization (G-IMLOP) to the anisotropic noise case are described. These algorithms may be understood as probabilistic variants of the popular iterative closest point (ICP) algorithm. A probabilistic model provides the framework, wherein both position information and orientation information are simultaneously optimized. Like ICP, the proposed algorithms iterate between correspondence and registration subphases. Efficient and optimal solutions are presented for implementing each subphase of the proposed methods. Experiments based on human femur data demonstrate that the IMLOP and G-IMLOP algorithms provide a strong accuracy advantage over ICP, with G-IMLOP providing additional accuracy improvement over IMLOP for registering data characterized by anisotropic noise. Furthermore, the proposed algorithms have increased ability to robustly identify an accurate versus inaccurate registration result. The IMLOP and G-IMLOP algorithms provide a cohesive framework for incorporating orientation data into the registration problem, thereby enabling improvement in accuracy as well as increased confidence in the quality of registration outcomes. For shape data having anisotropic uncertainty in position and/or orientation, the anisotropic noise model of G-IMLOP enables further gains in registration accuracy to be achieved.

Open-source image registration for MRI–TRUS fusion-guided prostate interventions

Abstract: Purpose We propose two software tools for non-rigid registration of MRI and transrectal ultrasound (TRUS) images of the prostate. Our ultimate goal is to develop an open-source solution to support MRI–TRUS fusion image guidance of prostate interventions, such as targeted biopsy for prostate cancer detection and focal therapy. It is widely hypothesized that image registration is an essential component in such systems.

Computer-assisted fracture reduction: a new approach for repositioning femoral fractures and planning reduction paths

Abstract: Reduction is a crucial step in the surgical treatment of bone fractures to achieve anatomical alignment and facilitate healing. Surgical planning for treatment of simple femoral fractures requires suitable gentle reduction paths. A plan with optimal movement of fracture fragments from the initial to the desired target position should improve the reduction procedure. A virtual environment which repositions the fracture fragments automatically and provides the ability to plan reduction paths was developed and tested. Virtual 3D osseous fragments are created from CT scans. Based on the computed surface curvatures, strongly curved edges are selected and fracture lines are generated. After assignment of matching points, the lines are compared and the desired target position is calculated. Planning of reduction paths was achieved using a reference-coordinate-system for the computation of reduction parameters. The fracture is reduced by changing the reduction parameters step by step until the target position is reached. To test this system, nine different fractured SYNBONE models and one human fracture were reduced, based on CT scans with varying resolution. The highest mean translational error is $$1.2 \pm 0.9$$ (mm), and the rotational error is $$2.6 \pm 2.8\, (^{\circ })$$ , both of which are considered as clinically acceptable. The reduction paths can be planned manually or semi-automatically for each fracture. Automated fracture reduction was achieved using a system based on preoperative CT scans. The automated system provides a clinically feasible basis for planning optimal reduction paths that may be augmented by further computer- or robot-assisted applications.

Robust surface tracking combining features, intensity and illumination compensation

Abstract: Purpose Recovering tissue deformation during robotic-assisted minimally invasive surgery procedures is important for providing intra-operative guidance, enabling in vivo imaging modalities and enhanced robotic control. The tissue motion can also be used to apply motion stabilization and to prescribe dynamic constraints for avoiding critical anatomical structures.

Automatic segmentation of the nasal cavity and paranasal sinuses from cone-beam CT images

Abstract: A patient-specific upper airway model is important for clinical, education, and research applications. Cone-beam computed tomography (CBCT) is used for imaging the upper airway but automatic segmentation is limited by noise and the complex anatomy. A multi-step level set segmentation scheme was developed for CBCT volumetric head scans to create a 3D model of the nasal cavity and paranasal sinuses. Gaussian mixture model thresholding and morphological operators are first employed to automatically locate the region of interest and to initialize the active contour. Second, the active contour driven by the Kullback–Leibler (K–L) divergence energy in a level set framework to segment the upper airway. The K–L divergence asymmetry is used to directly minimize the K–L divergence energy on the probability density function of the image intensity. Finally, to refine the segmentation result, an anisotropic localized active contour is employed which defines the local area based on shape prior information. The method was tested on ten CBCT data sets. The results were evaluated by the Dice coefficient, the volumetric overlap error (VOE), precision, recall, and $$F$$ -score and compared with expert manual segmentation and existing methods. The nasal cavity and paranasal sinuses were segmented in CBCT images with a median accuracy of 95.72 % [93.82–96.72 interquartile range] by Dice, 8.73 % [6.79–12.20] by VOE, 94.69 % [93.80–94.97] by precision, 97.73 % [92.70–98.79] by recall, and 95.72 % [93.82–96.69] by $$F$$ -score. Automated CBCT segmentation of the airway and paranasal sinuses was highly accurate in a test sample of clinical scans. The method may be useful in a variety of clinical, education, and research applications.

Robust camera localisation with depth reconstruction for bronchoscopic navigation

Abstract: Bronchoscopy is a standard technique for airway examination, providing a minimally invasive approach for both diagnosis and treatment of pulmonary diseases. To target lesions identified pre-operatively, it is necessary to register the location of the bronchoscope to the CT bronchial model during the examination. Existing vision-based techniques rely on the registration between virtually rendered endobronchial images and videos based on image intensity or surface geometry. However, intensity-based approaches are sensitive to illumination artefacts, while gradient-based approaches are vulnerable to surface texture. In this paper, depth information is employed in a novel way to achieve continuous and robust camera localisation. Surface shading has been used to recover depth from endobronchial images. The pose of the bronchoscopic camera is estimated by maximising the similarity between the depth recovered from a video image and that captured from a virtual camera projection of the CT model. The normalised cross-correlation and mutual information have both been used and compared for the similarity measure. The proposed depth-based tracking approach has been validated on both phantom and in vivo data. It outperforms the existing vision-based registration methods resulting in smaller pose estimation error of the bronchoscopic camera. It is shown that the proposed approach is more robust to illumination artefacts and surface texture and less sensitive to camera pose initialisation. A reliable camera localisation technique has been proposed based on depth information for bronchoscopic navigation. Qualitative and quantitative performance evaluations show the clinical value of the proposed framework.

Seeing is believing: increasing intraoperative awareness to scattered radiation in interventional procedures by combining augmented reality, Monte Carlo simulations and wireless dosimeters.

Abstract: Purpose Surgical staff performing image-guided minimally invasive surgical procedures are chronically exposed to harmful ionizing radiation. Currently, no means exist to intraoperatively depict the 3D shape and intensity of scattered radiation fields or to assess the body-part exposure of clinicians. We propose a system for simulating and visualizing intraoperative scattered radiation using augmented reality.

Locally rigid, vessel-based registration for laparoscopic liver surgery

Abstract: Purpose Laparoscopic liver resection has significant advantages over open surgery due to less patient trauma and faster recovery times, yet is difficult for most lesions due to the restricted field of view and lack of haptic feedback. Image guidance provides a potential solution but is challenging in a soft deforming organ such as the liver. In this paper, we therefore propose a laparoscopic ultrasound (LUS) image guidance system and study the feasibility of a locally rigid registration for laparoscopic liver surgery.

Development of customized positioning guides using computer-aided design and manufacturing technology for orthognathic surgery

Abstract: The purpose of this study was to devise a method for producing customized positioning guides for translating virtual plans to actual orthognathic surgery, and evaluation of the feasibility and validity of the devised method. Patients requiring two-jaw orthognathic surgery were enrolled and consented before operation. Two types of positioning guides were designed and fabricated using computer-aided design and manufacturing technology: One of the guides was used for the LeFort I osteotomy, and the other guide was used for positioning the maxillomandibular complex. The guides were fixed to the medial side of maxilla. For validation, the simulation images and postoperative cone beam computed tomography images were superimposed using surface registration to quantify the difference between the images. The data were presented in root-mean-square difference (RMSD) values. Both sets of guides were experienced to provide ideal fit and maximal contact to the maxillary surface to facilitate their accurate management in clinical applications. The validation results indicated that RMSD values between the images ranged from 0.18 to 0.33 mm in the maxilla and from 0.99 to 1.56 mm in the mandible. The patients were followed up for 6 months or more, and all of them were satisfied with the results. The proposed customized positioning guides are practical and reliable for translation of virtual plans to actual surgery. Furthermore, these guides improved the efficiency and outcome of surgery. This approach is uncomplicated in design, cost-effective in fabrication, and particularly convenient to use.