Vision-based and marker-less surgical tool detection and tracking: a review of the literature

Segmentation of the spinal column from computed tomography (CT) images is a preprocessing step for a range of image-guided interventions. One intervention that would benefit from accurate segmentation is spinal needle injection. Previous spinal segmentation techniques have primarily focused on identification and separate segmentation of each vertebra. Recently, statistical multi-object shape models have been introduced to extract common statistical characteristics between several anatomies. These models can be used for segmentation purposes because they are robust, accurate, and computationally tractable. In this paper, we develop a statistical multi-vertebrae shape+pose model and propose a novel registration-based technique to segment the CT images of spine. The multi-vertebrae statistical model captures the variations in shape and pose simultaneously, which reduces the number of registration parameters. We validate our technique in terms of accuracy and robustness of multi-vertebrae segmentation of CT images acquired from lumbar vertebrae of 32 subjects. The mean error of the proposed technique is below 2 mm, which is sufficient for many spinal needle injection procedures, such as facet joint injections.

Lumbar Spine Segmentation Using a Statistical Multi-Vertebrae Anatomical Shape+Pose Model

Monitoring vital signs through unobtrusive means is a goal which has attracted a lot of attention in the past decade. This review provides a systematic and comprehensive review over the current state of the field of ambient and unobtrusive cardiorespiratory monitoring. To this end, nine different sensing modalities which have been in the focus of current research activities are covered: capacitive electrocardiography, seismo- and ballistocardiography, reflective photoplethysmography (PPG) and PPG imaging, thermography, methods relying on laser or radar for distance-based measurements, video motion analysis, as well as methods using high-frequency electromagnetic fields. Current trends in these subfields are reviewed. Moreover, we systematically analyze similarities and differences between these methods with respect to the physiological and physical effects they sense as well as the resulting implications. Finally, future research trends for the field as a whole are identified.

Ambient and Unobtrusive Cardiorespiratory Monitoring Techniques

This article reviews the flexible robotic surgery and navigation technologies that are currently available and under research development, in particular for transoral robotic surgery, in both broad and narrow senses. The clinical background, classifications, associated biomedical robotics applications, and surgical outcomes are illustrated in this new paradigm of minimally invasive surgery. The state-of-the-art robotic and navigation systems for transoral procedures are reviewed by identifying their key properties and considerations. The use of different materials and actuation methods by current robotic systems offers various movements for different purposes, and their characteristics are compared. The future research trends of robotic and navigation systems for transoral procedures are discussed in terms of emerging new material, actuation, and sensing technologies.

Computer-assisted transoral surgery with flexible robotics and navigation technologies: a review of recent progress and research challenges.

Augmented reality (AR) has an increasing presence in the world of image-guided interventions which is amplified by the availability of consumer-grade head-mounted display (HMD) technology. The Microsoft® HoloLensTM optical passthrough device is at the forefront of consumer technology, as it is the first un-tethered head mounted computer (HMC). It shows promise of effectiveness in guiding clinical interventions, however its accuracy and stability must still be evaluated for the clinical environment. We have developed an evaluative protocol for the HoloLensTM using an optical measurement device to digitize the perceived pose of the rendered hologram. This evaluates the ability of the HoloLensTM to maintain the hologram in its intended pose. The stability is measured when actions are performed that may cause a shift in the holograms’ pose due to errors in its simultaneous localization and mapping. An emphasis is placed on actions that are more likely to be performed in a clinical setting. This will be used to determine the most applicable use cases for this technology in the future and how to minimize errors when in use. Our results show promise of this device’s potential for intraoperative clinical use. Further analysis must be performed to evaluate other potential sources of hologram disruption.

Hologram stability evaluation for Microsoft HoloLens

The technology for localization of surgical tools with respect to the patient's reference coordinate system in three to six degrees of freedom is one of the key components in computer aided surgery. Several tracking methods are available, of which optical tracking is the most widespread in clinical use. Optical tracking technology has proven to be a reliable method for intra-operative position and orientation acquisition in many clinical applications; however, the accuracy of such localizers is still a topic of discussion. In this paper, the accuracy of three optical localizer systems, the NDI Polaris P4, the NDI Polaris Spectra (in active and passive mode) and the Stryker Navigation System II camera, is assessed and compared critically. Static tests revealed that only the Polaris P4 shows significant warm-up behavior, with a significant shift of accuracy being observed within 42 minutes of being switched on. Furthermore, the intrinsic localizer accuracy was determined for single markers as well as for tools using a volumetric measurement protocol on a coordinate measurement machine. To determine the relative distance error within the measurement volume, the Length Measurement Error (LME) was determined at 35 test lengths. As accuracy depends strongly on the marker configuration employed, the error to be expected in typical clinical setups was estimated in a simulation for different tool configurations. The two active localizer systems, the Stryker Navigation System II camera and the Polaris Spectra (active mode), showed the best results, with trueness values (mean +/- standard deviation) of 0.058 +/- 0.033 mm and 0.089 +/- 0.061 mm, respectively. The Polaris Spectra (passive mode) showed a trueness of 0.170 +/- 0.090 mm, and the Polaris P4 showed the lowest trueness at 0.272 +/- 0.394 mm with a higher number of outliers than for the other cameras. The simulation of the different tool configurations in a typical clinical setup revealed that the tracking error can be estimated to be 1.02 mm for the Polaris P4, 0.64 mm for the Polaris Spectra in passive mode, 0.33 mm for the Polaris Spectra in active mode, and 0.22 mm for the Stryker Navigation System II camera.

https://www.tandfonline.com/doi/pdf/10.3109/10929081003647239

Assessment of optical localizer accuracy for computer aided surgery systems

An electromagnetic-based tracking and navigation system was evaluated for interventional radiology. The electromagnetic tracking system (CAPPA IRAD EMT, CASinnovations, Erlangen, Germany) was used for real-time monitoring of punctures of the lumbar facet joints and intervertebral disks in a spine phantom, three pig cadavers and three anaesthesized pigs. Therefore, pre-interventional computed tomography (CT) datasets were transferred to the navigation system and puncture trajectories were planned. A coaxial needle was advanced along the trajectories while the position of the needle tip was monitored in real time. After puncture tracts were marked with pieces of wire another CT examination was performed and distances between wires and anatomical targets were measured. Performing punctures of the facet joints mean needle positioning errors were 0.4 ± 0.8 mm in the spine phantom, 2.8 ± 2.1 mm ex vivo and 3.0 ± 2.0 mm in vivo with mean length of the puncture tract of 54.0 ± 10.4 mm (phantom), 51.6 ± 12.6 mm (ex vivo) and 50.9 ± 17.6 mm (in vivo). At first attempt, intervertebral discs were successfully punctured in 15/15 in the phantom study, in 12/15 in the ex-vivo study and 14/15 in the in-vivo study, respectively. Immobilization of the patient and optimal positioning of the field generator are essential to achieve a high accuracy of needle placement in a clinical CT setting.

Electromagnetic tracking for CT-guided spine interventions: phantom, ex-vivo and in-vivo results.

On the way to a cable free operating theater a new operating table is developed featuring into the table integrated sensors for patient monitoring, which do not necessarily need a direct contact to the naked skin, and a magnetic tracking system. The patient monitoring sensors are new multimodal sensors, which combine a capacitive electric field sensor with a reflective optical sensor and a temperature measurement. This multimodal measurement offers the opportunity of a reliable vital sign measurement by use of redundancy and plausibility checks. A novel artifact detection method for capacitive electrocardiogram measurements based on an optical measurement is presented. Together with an adaptive threshold based algorithm, intervals with artifacts can reliably be identified resulting in a robust estimation of heart rate.

/pdf/on-the-way-to-a-cable-free-operating-theater-an-operating-1bdcnfynq7.pdf

On the way to a cable free operating theater: An operating table with integrated multimodal monitoring

Preservation and recovery of the mechanical leg axis as well as good rotational alignment of the prosthesis components and well-balanced ligaments are essential for the longevity of total knee arthroplasty (TKA). In the framework of the OrthoMIT project, the genALIGN system, a new navigated implantation approach based on intra-operative force-torque measurements, has been developed. With this system, optical or magnetic position tracking as well as any fixation of invasive rigid bodies are no longer necessary. For the alignment of the femoral component along the mechanical axis, a sensor-integrated instrument measures the torques resulting from the deviation between the instrument's axis and the mechanical axis under manually applied axial compression load. When both axes are coaxial, the resulting torques equal zero, and the tool axis can be fixed with respect to the bone. For ligament balancing and rotational alignment of the femoral component, the genALIGN system comprises a sensor-integrated tibial trial inlay measuring the amplitude and application points of the forces transferred between femur and tibia. Hereby, the impact of ligament tensions on knee joint loads can be determined over the whole range of motion. First studies with the genALIGN system, including a comparison with an imageless navigation system, show the feasibility of the concept.

A new approach to implant alignment and ligament balancing in total knee arthroplasty focussing on joint loads.

This study assesses the feasibility of navigated vascular interventions in a rapid prototyped anthropomorphic phantom and swine cadaveric model using electromagnetic tracking (EMT) in combination with a CT image data set. In the phantom model overall feasibility and handling of the EMT navigation system were evaluated and first experiments performing visceral vessel catheterization were carried out. In the cadaveric model a predefined structure was targeted using an electromagnetically tracked guidewire. In the phantom model the catheterization of visceral vessels was reproducibly possible. In the cadaveric model a catheter placement accuracy of 0.97 ± 0.88 mm was achieved. The feasibility of EMT based vascular interventions could be shown and a landmark concerning the accuracy of such systems in a model setting could be determined. Further ex- and in-vivo experiments are needed to preevaluate applicability of such systems in clinical settings.

Robert Elfring

Papers

Assessment of optical localizer accuracy for computer aided surgery systems

Electromagnetic tracking for CT-guided spine interventions: phantom, ex-vivo and in-vivo results.

On the way to a cable free operating theater: An operating table with integrated multimodal monitoring

A new approach to implant alignment and ligament balancing in total knee arthroplasty focussing on joint loads.

Vascular electromagnetic tracking: experiences in phantom and animal cadaveric models