The quality of oral and maxillofacial surgery (OMS) significantly depends on the accuracy of surgical navigation. In this article, a vision-based markerless surgical navigation system is developed to overcome the shortcomings in the currently available technologies. Registration methods both for patient and surgical instrument tracking are improved to increase the navigation performance. For patient-image registration, we propose an efficient texture-less pose estimation method using only shape information. An innovative strategy is developed to effectively reject the outliers and improve the pose accuracy, which is the first attempt at introducing geometric matching information to guide PnP calculation. For surgical instrument tracking, a position-sensing marker is used to achieve robust and convenient instrument localization with high accuracy. Experiments were conducted on the 3-D-printed maxilla and mandible models to evaluate the navigation performance. Evaluation results validate the effectiveness of the proposed pose estimation method in improving the pose accuracy for texture-less teeth. Besides, it is revealed that the position-sensing marker can be localized with high accuracy even under nonideal visibility conditions, which expands the motion range of the instrument and decreases the size of the tool. The entire system has a sufficiently small target registration error (TRE). These experimental results have verified that the proposed surgical navigation system can provide practical guidance for OMS with satisfactory accuracy. 

A Vision-Based Navigation System With Markerless Image Registration and Position-Sensing Localization for Oral and Maxillofacial Surgery

Visuotactile sensing technology has received much attention in recent years. This article proposes a feature detection method applicable to visuotactile sensors based on continuous marker patterns (CMP) to measure 3-d deformation. First, we construct the feature model of checkerboard-like corners under contact deformation, and design a novel double-layer circular sampler. Then, we propose the judging criteria and response function of corner features by analyzing sampling signals' amplitude-frequency characteristics and circular cross-correlation behavior. The proposed feature detection algorithm fully considers the boundary characteristics retained by the corners with geometric distortion, thus enabling reliable detection at a low calculation cost. The experimental results show that the proposed method has significant advantages in terms of real-time and robustness. Finally, we have achieved the high-density 3-d contact deformation visualization based on this detection method. This technique is able to clearly record the process of contact deformation, thus enabling inverse sensing of dynamic contact processes.

Real-time and Robust Feature Detection of Continuous Marker Pattern for Dense 3-D Deformation Measurement

The quality of oral and maxillofacial surgery (OMS) significantly depends on the accuracy of surgical navigation. In this article, a vision-based markerless surgical navigation system is developed to overcome the shortcomings in the currently available technologies. Registration methods both for patient and surgical instrument tracking are improved to increase the navigation performance. For patient-image registration, we propose an efficient texture-less pose estimation method using only shape information. An innovative strategy is developed to effectively reject the outliers and improve the pose accuracy, which is the first attempt at introducing geometric matching information to guide PnP calculation. For surgical instrument tracking, a position-sensing marker is used to achieve robust and convenient instrument localization with high accuracy. Experiments were conducted on the 3-D-printed maxilla and mandible models to evaluate the navigation performance. Evaluation results validate the effectiveness of the proposed pose estimation method in improving the pose accuracy for texture-less teeth. Besides, it is revealed that the position-sensing marker can be localized with high accuracy even under nonideal visibility conditions, which expands the motion range of the instrument and decreases the size of the tool. The entire system has a sufficiently small target registration error (TRE). These experimental results have verified that the proposed surgical navigation system can provide practical guidance for OMS with satisfactory accuracy.

Image-guided navigation system for minimally invasive total hip arthroplasty (MITHA) using an improved position-sensing marker.

Checkerboard-like markers are widely applied in visual localization applications, including SLAM, augmented reality, robot navigation, and 3-D scene reconstruction. Most corner detectors assume that the marker is attached to a flat surface, which severely limits the placement of the marker and thus reduces the scope of use for these applications. However, in some scenarios, it is not easy to find an ideal flat surface required by most corner detectors in a given scenario, in which case the marker can often only be fixed on a curved surface. Therefore, the accuracy of most corner detectors may be reduced. In this study, a novel method is proposed with subpixel accuracy to detect and locate the corners on a chessboard-like marker, which is either flat or curved. The proposed method fits multisegment curves in a quadratic form to the edges in a checkerboard-like marker. The exact corner positions are considered as the intersections of the corresponding curves with analytical solutions. The proposed method achieves the state-of-the-art performance through experiments, including synthetic corner localization test, real-world stereo vision triangulation experiment, and pose estimation on a curved object, demonstrating the superiority of our method.

Accurate Detection and Localization of Curved Checkerboard-Like Marker Based on Quadratic Form

This paper presents a novel lightweight navigation system for oral and maxillofacial surgery (OMS). An external curved checkerboard with self-identifying markers is set as the reference object around the surgical scene. A customized oral clip with a micro camera is designed for oral localization by tracking the external checkerboard. Similarly, the dental handpiece is also equipped with a micro camera, which can be localized like the clip. The spatial model of the markers is provided by binocular stereoscopic reconstruction. A front surface mirror is taken up for the registration between the oral cavity and the camera on the clip. The pivot calibration of the dental handpiece is accomplished by our proposed calibration method. We set up an experimental group and a control group for evaluation. The surgical tools of the experimental group were approximately 30% lighter, 35% less bulky, and 90% cheaper than those of the control group. Our system yielded the comprehensive navigation accuracy of 0.92 mm whereas the accuracy of the control group was 0.87 mm. Results revealed that our system can achieve similar accuracy compared with a prevailing system at a lighter weight, a more compact volume, and a lower cost. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this work is to reduce the burden on patients and surgeons during OMS. Current commercial navigation systems are still limited by the burden of extra cumbersome fiducial markers and high hardware costs. Their high accuracy benefits from the large size of fiducial markers. To take full advantage of the camera’s localization effect, we propose the concept of “marker-camera inverse projection”, i.e., reversing the roles of the camera and the markers. In this way, cameras on surgical tools detect more points with a more uniform distribution. Our proposed system achieves a decent balance between navigation accuracy and hardware cost, which facilitates the development of surgical tools to be lighter and more economical, and involves tremendous potential for commercialization. 

Dongyue Li

Papers

Accurate Detection and Localization of Curved Checkerboard-Like Marker Based on Quadratic Form

A Novel Lightweight Navigation System for Oral and Maxillofacial Surgery Using an External Curved Self-Identifying Checkerboard

A Vision-Based Navigation System With Markerless Image Registration and Position-Sensing Localization for Oral and Maxillofacial Surgery