Showing papers in "International Journal of Computer Assisted Radiology and Surgery in 2017"

Automatic 3D liver location and segmentation via convolutional neural network and graph cut

Abstract: Segmentation of the liver from abdominal computed tomography (CT) images is an essential step in some computer-assisted clinical interventions, such as surgery planning for living donor liver transplant, radiotherapy and volume measurement. In this work, we develop a deep learning algorithm with graph cut refinement to automatically segment the liver in CT scans. The proposed method consists of two main steps: (i) simultaneously liver detection and probabilistic segmentation using 3D convolutional neural network; (ii) accuracy refinement of the initial segmentation with graph cut and the previously learned probability map. The proposed approach was validated on forty CT volumes taken from two public databases MICCAI-Sliver07 and 3Dircadb1. For the MICCAI-Sliver07 test dataset, the calculated mean ratios of volumetric overlap error (VOE), relative volume difference (RVD), average symmetric surface distance (ASD), root-mean-square symmetric surface distance (RMSD) and maximum symmetric surface distance (MSD) are 5.9, 2.7 %, 0.91, 1.88 and 18.94 mm, respectively. For the 3Dircadb1 dataset, the calculated mean ratios of VOE, RVD, ASD, RMSD and MSD are 9.36, 0.97 %, 1.89, 4.15 and 33.14 mm, respectively. The proposed method is fully automatic without any user interaction. Quantitative results reveal that the proposed approach is efficient and accurate for hepatic volume estimation in a clinical setup. The high correlation between the automatic and manual references shows that the proposed method can be good enough to replace the time-consuming and nonreproducible manual segmentation method.

Automated brain tumour detection and segmentation using superpixel-based extremely randomized trees in FLAIR MRI.

Abstract: We propose a fully automated method for detection and segmentation of the abnormal tissue associated with brain tumour (tumour core and oedema) from Fluid- Attenuated Inversion Recovery (FLAIR) Magnetic Resonance Imaging (MRI). The method is based on superpixel technique and classification of each superpixel. A number of novel image features including intensity-based, Gabor textons, fractal analysis and curvatures are calculated from each superpixel within the entire brain area in FLAIR MRI to ensure a robust classification. Extremely randomized trees (ERT) classifier is compared with support vector machine (SVM) to classify each superpixel into tumour and non-tumour. The proposed method is evaluated on two datasets: (1) Our own clinical dataset: 19 MRI FLAIR images of patients with gliomas of grade II to IV, and (2) BRATS 2012 dataset: 30 FLAIR images with 10 low-grade and 20 high-grade gliomas. The experimental results demonstrate the high detection and segmentation performance of the proposed method using ERT classifier. For our own cohort, the average detection sensitivity, balanced error rate and the Dice overlap measure for the segmented tumour against the ground truth are 89.48 %, 6 % and 0.91, respectively, while, for the BRATS dataset, the corresponding evaluation results are 88.09 %, 6 % and 0.88, respectively. This provides a close match to expert delineation across all grades of glioma, leading to a faster and more reproducible method of brain tumour detection and delineation to aid patient management.

Pulmonary nodule classification with deep residual networks

Abstract: Lung cancer has the highest death rate among all cancers in the USA. In this work we focus on improving the ability of computer-aided diagnosis (CAD) systems to predict the malignancy of nodules from cropped CT images of lung nodules. We evaluate the effectiveness of very deep convolutional neural networks at the task of expert-level lung nodule malignancy classification. Using the state-of-the-art ResNet architecture as our basis, we explore the effect of curriculum learning, transfer learning, and varying network depth on the accuracy of malignancy classification. Due to a lack of public datasets with standardized problem definitions and train/test splits, studies in this area tend to not compare directly against other existing work. This makes it hard to know the relative improvement in the new solution. In contrast, we directly compare our system against two state-of-the-art deep learning systems for nodule classification on the LIDC/IDRI dataset using the same experimental setup and data set. The results show that our system achieves the highest performance in terms of all metrics measured including sensitivity, specificity, precision, AUROC, and accuracy. The proposed method of combining deep residual learning, curriculum learning, and transfer learning translates to high nodule classification accuracy. This reveals a promising new direction for effective pulmonary nodule CAD systems that mirrors the success of recent deep learning advances in other image-based application domains.

Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets

Abstract: Multi-organ segmentation from CT images is an essential step for computer-aided diagnosis and surgery planning. However, manual delineation of the organs by radiologists is tedious, time-consuming and poorly reproducible. Therefore, we propose a fully automatic method for the segmentation of multiple organs from three-dimensional abdominal CT images. The proposed method employs deep fully convolutional neural networks (CNNs) for organ detection and segmentation, which is further refined by a time-implicit multi-phase evolution method. Firstly, a 3D CNN is trained to automatically localize and delineate the organs of interest with a probability prediction map. The learned probability map provides both subject-specific spatial priors and initialization for subsequent fine segmentation. Then, for the refinement of the multi-organ segmentation, image intensity models, probability priors as well as a disjoint region constraint are incorporated into an unified energy functional. Finally, a novel time-implicit multi-phase level-set algorithm is utilized to efficiently optimize the proposed energy functional model. Our method has been evaluated on 140 abdominal CT scans for the segmentation of four organs (liver, spleen and both kidneys). With respect to the ground truth, average Dice overlap ratios for the liver, spleen and both kidneys are 96.0, 94.2 and 95.4%, respectively, and average symmetric surface distance is less than 1.3 mm for all the segmented organs. The computation time for a CT volume is 125 s in average. The achieved accuracy compares well to state-of-the-art methods with much higher efficiency. A fully automatic method for multi-organ segmentation from abdominal CT images was developed and evaluated. The results demonstrated its potential in clinical usage with high effectiveness, robustness and efficiency.

Multi-level deep supervised networks for retinal vessel segmentation.

Abstract: Changes in the appearance of retinal blood vessels are an important indicator for various ophthalmologic and cardiovascular diseases, including diabetes, hypertension, arteriosclerosis, and choroidal neovascularization Vessel segmentation from retinal images is very challenging because of low blood vessel contrast, intricate vessel topology, and the presence of pathologies such as microaneurysms and hemorrhages To overcome these challenges, we propose a neural network-based method for vessel segmentation A deep supervised fully convolutional network is developed by leveraging multi-level hierarchical features of the deep networks To improve the discriminative capability of features in lower layers of the deep network and guide the gradient back propagation to overcome gradient vanishing, deep supervision with auxiliary classifiers is incorporated in some intermediate layers of the network Moreover, the transferred knowledge learned from other domains is used to alleviate the issue of insufficient medical training data The proposed approach does not rely on hand-crafted features and needs no problem-specific preprocessing or postprocessing, which reduces the impact of subjective factors We evaluate the proposed method on three publicly available databases, the DRIVE, STARE, and CHASE_DB1 databases Extensive experiments demonstrate that our approach achieves better or comparable performance to state-of-the-art methods with a much faster processing speed, making it suitable for real-world clinical applications The results of cross-training experiments demonstrate its robustness with respect to the training set The proposed approach segments retinal vessels accurately with a much faster processing speed and can be easily applied to other biomedical segmentation tasks

Breast ultrasound image segmentation: a survey

Abstract: Breast cancer is the most common form of cancer among women worldwide. Ultrasound imaging is one of the most frequently used diagnostic tools to detect and classify abnormalities of the breast. Recently, computer-aided diagnosis (CAD) systems using ultrasound images have been developed to help radiologists to increase diagnosis accuracy. However, accurate ultrasound image segmentation remains a challenging problem due to various ultrasound artifacts. In this paper, we investigate approaches developed for breast ultrasound (BUS) image segmentation. In this paper, we reviewed the literature on the segmentation of BUS images according to the techniques adopted, especially over the past 10 years. By dividing into seven classes (i.e., thresholding-based, clustering-based, watershed-based, graph-based, active contour model, Markov random field and neural network), we have introduced corresponding techniques and representative papers accordingly. We have summarized and compared many techniques on BUS image segmentation and found that all these techniques have their own pros and cons. However, BUS image segmentation is still an open and challenging problem due to various ultrasound artifacts introduced in the process of imaging, including high speckle noise, low contrast, blurry boundaries, low signal-to-noise ratio and intensity inhomogeneity To the best of our knowledge, this is the first comprehensive review of the approaches developed for segmentation of BUS images. With most techniques involved, this paper will be useful and helpful for researchers working on segmentation of ultrasound images, and for BUS CAD system developers.

Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks

Abstract: Delineation of thyroid nodule boundaries from ultrasound images plays an important role in calculation of clinical indices and diagnosis of thyroid diseases. However, it is challenging for accurate and automatic segmentation of thyroid nodules because of their heterogeneous appearance and components similar to the background. In this study, we employ a deep convolutional neural network (CNN) to automatically segment thyroid nodules from ultrasound images. Our CNN-based method formulates a thyroid nodule segmentation problem as a patch classification task, where the relationship among patches is ignored. Specifically, the CNN used image patches from images of normal thyroids and thyroid nodules as inputs and then generated the segmentation probability maps as outputs. A multi-view strategy is used to improve the performance of the CNN-based model. Additionally, we compared the performance of our approach with that of the commonly used segmentation methods on the same dataset. The experimental results suggest that our proposed method outperforms prior methods on thyroid nodule segmentation. Moreover, the results show that the CNN-based model is able to delineate multiple nodules in thyroid ultrasound images accurately and effectively. In detail, our CNN-based model can achieve an average of the overlap metric, dice ratio, true positive rate, false positive rate, and modified Hausdorff distance as $$0.8683 \pm 0.0056$$ , $$0.9224 \pm 0.0027$$ , $$0.915 \pm 0.0077$$ , $$0.0669 \pm 0.0032$$ , $$0.6228 \pm 0.1414$$ on overall folds, respectively. Our proposed method is fully automatic without any user interaction. Quantitative results also indicate that our method is so efficient and accurate that it can be good enough to replace the time-consuming and tedious manual segmentation approach, demonstrating the potential clinical applications.

Fabricating biomedical origami: a state-of-the-art review

Abstract: Purpose Origami-based biomedical device design is an emerging technology due to its ability to be deployed from a minimal foldable pattern to a larger volume. This paper aims to review state-of-the-art origami structures applied in the medical device field.

Touchless interaction with software in interventional radiology and surgery: a systematic literature review

Abstract: In this article, we systematically examine the current state of research of systems that focus on touchless human–computer interaction in operating rooms and interventional radiology suites. We further discuss the drawbacks of current solutions and underline promising technologies for future development. A systematic literature search of scientific papers that deal with touchless control of medical software in the immediate environment of the operation room and interventional radiology suite was performed. This includes methods for touchless gesture interaction, voice control and eye tracking. Fifty-five research papers were identified and analyzed in detail including 33 journal publications. Most of the identified literature (62 %) deals with the control of medical image viewers. The others present interaction techniques for laparoscopic assistance (13 %), telerobotic assistance and operating room control (9 % each) as well as for robotic operating room assistance and intraoperative registration (3.5 % each). Only 8 systems (14.5 %) were tested in a real clinical environment, and 7 (12.7 %) were not evaluated at all. In the last 10 years, many advancements have led to robust touchless interaction approaches. However, only a few have been systematically evaluated in real operating room settings. Further research is required to cope with current limitations of touchless software interfaces in clinical environments. The main challenges for future research are the improvement and evaluation of usability and intuitiveness of touchless human–computer interaction and the full integration into productive systems as well as the reduction of necessary interaction steps and further development of hands-free interaction.

Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study

Abstract: We present a novel augmented reality (AR) surgical navigation system based on ultrasound-assisted registration for pedicle screw placement. This system provides the clinically desired targeting accuracy and reduces radiation exposure. Ultrasound (US) is used to perform registration between preoperative computed tomography (CT) images and patient, and the registration is performed by least-squares fitting of these two three-dimensional (3D) point sets of anatomical landmarks taken from US and CT images. An integral videography overlay device is calibrated to accurately display naked-eye 3D images for surgical navigation. We use a 3.0-mm Kirschner wire (K-wire) instead of a pedicle screw in this study, and the K-wire is calibrated to obtain its orientation and tip location. Based on the above registration and calibration, naked-eye 3D images of the planning path and the spine are superimposed onto patient in situ using our AR navigation system. Simultaneously, a 3D image of the K-wire is overlaid accurately on the real one to guide the insertion procedure. The targeting accuracy is evaluated postoperatively by performing a CT scan. An agar phantom experiment was performed. Eight K-wires were inserted successfully after US-assisted registration, and the mean targeting error and angle error were 3.35 mm and $$2.74{^{\circ }}$$ , respectively. Furthermore, an additional sheep cadaver experiment was performed. Four K-wires were inserted successfully. The mean targeting error was 3.79 mm and the mean angle error was $$4.51{^{\circ }}$$ , and US-assisted registration yielded better targeting results than skin markers-based registration (targeting errors: 2.41 vs. 5.18 mm, angle errors: $$3.13{^{\circ }}$$ vs. $$5.89{^{\circ }})$$ . Experimental outcomes demonstrate that the proposed navigation system has acceptable targeting accuracy. In particular, the proposed navigation method reduces repeated radiation exposure to the patient and surgeons. Therefore, it has promising prospects for clinical use.

Comparison of optical see-through head-mounted displays for surgical interventions with object-anchored 2D-display.

Abstract: Optical see-through head-mounted displays (OST-HMD) feature an unhindered and instantaneous view of the surgery site and can enable a mixed reality experience for surgeons during procedures. In this paper, we present a systematic approach to identify the criteria for evaluation of OST-HMD technologies for specific clinical scenarios, which benefit from using an object-anchored 2D-display visualizing medical information. Criteria for evaluating the performance of OST-HMDs for visualization of medical information and its usage are identified and proposed. These include text readability, contrast perception, task load, frame rate, and system lag. We choose to compare three commercially available OST-HMDs, which are representatives of currently available head-mounted display technologies. A multi-user study and an offline experiment are conducted to evaluate their performance. Statistical analysis demonstrates that Microsoft HoloLens performs best among the three tested OST-HMDs, in terms of contrast perception, task load, and frame rate, while ODG R-7 offers similar text readability. The integration of indoor localization and fiducial tracking on the HoloLens provides significantly less system lag in a relatively motionless scenario. With ever more OST-HMDs appearing on the market, the proposed criteria could be used in the evaluation of their suitability for mixed reality surgical intervention. Currently, Microsoft HoloLens may be more suitable than ODG R-7 and Epson Moverio BT-200 for clinical usability in terms of the evaluated criteria. To the best of our knowledge, this is the first paper that presents a methodology and conducts experiments to evaluate and compare OST-HMDs for their use as object-anchored 2D-display during interventions.

Cost-effective, personalized, 3D-printed liver model for preoperative planning before laparoscopic liver hemihepatectomy for colorectal cancer metastases

Abstract: Three-dimensional (3D) printing for preoperative planning has been intensively developed in the recent years. However, the implementation of these solutions in hospitals is still difficult due to high costs, extremely expensive industrial-grade printers, and software that is difficult to obtain and learn along with a lack of a defined process. This paper presents a cost-effective technique of preparing 3D-printed liver models that preserves the shape and all of the structures, including the vessels and the tumor, which in the present case is colorectal liver metastasis. The patient’s computed tomography scans were used for the separation and visualization of virtual 3D anatomical structures. Those elements were transformed into stereolithographic files and subsequently printed on a desktop 3D printer. The multipart structure was assembled and filled with silicone. The patient underwent subsequent laparoscopic right hemihepatectomy. The entire process is described step-by-step, and only free-to-use and mostly open-source software was used. As a result, a transparent, full-sized liver model with visible vessels and colorectal metastasis was created for under $150, which—taking into account 3D printer prices—is much cheaper than models presented in previous research papers. The increased accessibility of 3D models for physicians before complex laparoscopic surgical procedures such as hepatic resections could lead to beneficial breakthroughs in these sophisticated surgeries, as many reports show that these models reduce operative time and improve short term outcomes.

Extraction of skin lesions from non-dermoscopic images for surgical excision of melanoma.

Abstract: Computerized prescreening of suspicious moles and lesions for malignancy is of great importance for assessing the need and the priority of the removal surgery. Detection can be done by images captured by standard cameras, which are more preferable due to low cost and availability. One important step in computerized evaluation is accurate detection of lesion’s region, i.e., segmentation of an image into two regions as lesion and normal skin. In this paper, a new method based on deep neural networks is proposed for accurate extraction of a lesion region. The input image is preprocessed, and then, its patches are fed to a convolutional neural network. Local texture and global structure of the patches are processed in order to assign pixels to lesion or normal classes. A method for effective selection of training patches is proposed for more accurate detection of a lesion’s border. Our results indicate that the proposed method could reach the accuracy of 98.7% and the sensitivity of 95.2% in segmentation of lesion regions over the dataset of clinical images. The experimental results of qualitative and quantitative evaluations demonstrate that our method can outperform other state-of-the-art algorithms exist in the literature.

IBIS: an OR ready open-source platform for image-guided neurosurgery

Abstract: Navigation systems commonly used in neurosurgery suffer from two main drawbacks: (1) their accuracy degrades over the course of the operation and (2) they require the surgeon to mentally map images from the monitor to the patient. In this paper, we introduce the Intraoperative Brain Imaging System (IBIS), an open-source image-guided neurosurgery research platform that implements a novel workflow where navigation accuracy is improved using tracked intraoperative ultrasound (iUS) and the visualization of navigation information is facilitated through the use of augmented reality (AR). The IBIS platform allows a surgeon to capture tracked iUS images and use them to automatically update preoperative patient models and plans through fast GPU-based reconstruction and registration methods. Navigation, resection and iUS-based brain shift correction can all be performed using an AR view. IBIS has an intuitive graphical user interface for the calibration of a US probe, a surgical pointer as well as video devices used for AR (e.g., a surgical microscope). The components of IBIS have been validated in the laboratory and evaluated in the operating room. Image-to-patient registration accuracy is on the order of $$3.72\pm 1.27\,\hbox {mm}$$ and can be improved with iUS to a median target registration error of 2.54 mm. The accuracy of the US probe calibration is between 0.49 and 0.82 mm. The average reprojection error of the AR system is $$0.37\pm 0.19\,\hbox {mm}$$ . The system has been used in the operating room for various types of surgery, including brain tumor resection, vascular neurosurgery, spine surgery and DBS electrode implantation. The IBIS platform is a validated system that allows researchers to quickly bring the results of their work into the operating room for evaluation. It is the first open-source navigation system to provide a complete solution for AR visualization.

On-patient see-through augmented reality based on visual SLAM.

Abstract: An augmented reality system to visualize a 3D preoperative anatomical model on intra-operative patient is proposed. The hardware requirement is commercial tablet-PC equipped with a camera. Thus, no external tracking device nor artificial landmarks on the patient are required. We resort to visual SLAM to provide markerless real-time tablet-PC camera location with respect to the patient. The preoperative model is registered with respect to the patient through 4–6 anchor points. The anchors correspond to anatomical references selected on the tablet-PC screen at the beginning of the procedure. Accurate and real-time preoperative model alignment (approximately 5-mm mean FRE and TRE) was achieved, even when anchors were not visible in the current field of view. The system has been experimentally validated on human volunteers, in vivo pigs and a phantom. The proposed system can be smoothly integrated into the surgical workflow because it: (1) operates in real time, (2) requires minimal additional hardware only a tablet-PC with camera, (3) is robust to occlusion, (4) requires minimal interaction from the medical staff.

Deep monocular 3D reconstruction for assisted navigation in bronchoscopy

Abstract: In bronchoschopy, computer vision systems for navigation assistance are an attractive low-cost solution to guide the endoscopist to target peripheral lesions for biopsy and histological analysis. We propose a decoupled deep learning architecture that projects input frames onto the domain of CT renderings, thus allowing offline training from patient-specific CT data. A fully convolutional network architecture is implemented on GPU and tested on a phantom dataset involving 32 video sequences and $$\sim $$ 60k frames with aligned ground truth and renderings, which is made available as the first public dataset for bronchoscopy navigation. An average estimated depth accuracy of 1.5 mm was obtained, outperforming conventional direct depth estimation from input frames by 60%, and with a computational time of $$\le $$ 30 ms on modern GPUs. Qualitatively, the estimated depth and renderings closely resemble the ground truth. The proposed method shows a novel architecture to perform real-time monocular depth estimation without losing patient specificity in bronchoscopy. Future work will include integration within SLAM systems and collection of in vivo datasets.

SLIDE: automatic spine level identification system using a deep convolutional neural network.

Abstract: Percutaneous spinal needle insertion procedures often require proper identification of the vertebral level to effectively and safely deliver analgesic agents. The current clinical method involves “blind” identification of the vertebral level through manual palpation of the spine, which has only 30% reported accuracy. Therefore, there is a need for better anatomical identification prior to needle insertion. A real-time system was developed to identify the vertebral level from a sequence of ultrasound images, following a clinical imaging protocol. The system uses a deep convolutional neural network (CNN) to classify transverse images of the lower spine. Several existing CNN architectures were implemented, utilizing transfer learning, and compared for adequacy in a real-time system. In the system, the CNN output is processed, using a novel state machine, to automatically identify vertebral levels as the transducer moves up the spine. Additionally, a graphical display was developed and integrated within 3D Slicer. Finally, an augmented reality display, projecting the level onto the patient’s back, was also designed. A small feasibility study $$(n=20)$$ evaluated performance. The proposed CNN successfully discriminates ultrasound images of the sacrum, intervertebral gaps, and vertebral bones, achieving 88% 20-fold cross-validation accuracy. Seventeen of 20 test ultrasound scans had successful identification of all vertebral levels, processed at real-time speed (40 frames/s). A machine learning system is presented that successfully identifies lumbar vertebral levels. The small study on human subjects demonstrated real-time performance. A projection-based augmented reality display was used to show the vertebral level directly on the subject adjacent to the puncture site.

Real-time surgical tool tracking and pose estimation using a hybrid cylindrical marker

Abstract: To provide an integrated visualisation of intraoperative ultrasound and endoscopic images to facilitate intraoperative guidance, real-time tracking of the ultrasound probe is required. State-of-the-art methods are suitable for planar targets while most of the laparoscopic ultrasound probes are cylindrical objects. A tracking framework for cylindrical objects with a large work space will improve the usability of the intraoperative ultrasound guidance. A hybrid marker design that combines circular dots and chessboard vertices is proposed for facilitating tracking cylindrical tools. The circular dots placed over the curved surface are used for pose estimation. The chessboard vertices are employed to provide additional information for resolving the ambiguous pose problem due to the use of planar model points under a monocular camera. Furthermore, temporal information between consecutive images is considered to minimise tracking failures with real-time computational performance. Detailed validation confirms that our hybrid marker provides a large working space for different tool sizes (6–14 mm in diameter). The tracking framework allows translational movements between 40 and 185 mm along the depth direction and rotational motion around three local orthogonal axes up to $$ \pm 80^\circ $$ . Comparative studies with the current state of the art confirm that our approach outperforms existing methods by providing nearly 100% detection rates and accurate pose estimation with mean errors of 2.8 mm and 0.72 $$^\circ $$ . The tracking algorithm runs at 20 frames per second for $$960\times 540$$ image resolution videos. Experiments show that the proposed hybrid marker can be applied to a wide range of surgical tools with superior detection rates and pose estimation accuracies. Both the qualitative and quantitative results demonstrate that our framework can be used not only for assisting intraoperative ultrasound guidance but also for tracking general surgical tools in MIS.

2D versus 3D fluoroscopy-based navigation in posterior pelvic fixation: review of the literature on current technology.

Abstract: Percutaneous sacroiliac (SI) fixation of unstable posterior pelvic ring injuries is a widely accepted procedure. The complex sacral anatomy with narrow osseous corridors for SI screw placement makes this procedure technically challenging. Techniques are constantly evolving as a result of better understanding of the posterior pelvic anatomy. Recently developed tools include fluoroscopy-based computer-assisted navigation, which can be two-dimensional (2D) or three-dimensional (3D). Our goal is to determine the relevant technical considerations and clinical outcomes associated with these modalities by reviewing the published research. We hypothesize that 3D fluoroscopy-based navigation is safer and superior to its 2D predecessor with respect to lower radiation dose and more accurate SI screw placement. We searched four medical databases to identify English-language studies of 2D and 3D fluoroscopy-based navigation from January 1990 through August 2015. We included articles reporting imaging techniques and outcomes of closed posterior pelvic ring fixation with percutaneous SI screw fixation. Injuries included in the study were sacral fractures (52 patients), sacroiliac fractures (88 patients), lateral compression fractures (20 patients), and anteroposterior compression type pelvic fractures (8 patients). We excluded articles on open reduction of posterior pelvic ring injuries and solely anatomic studies. We then reviewed these studies for technical considerations and outcomes associated with these technologies. Six studies were included in our analysis. Results of these studies indicate that 3D fluoroscopy-based navigation is associated with a lower radiation dose and lower rate of screw malpositioning compared with 2D fluoroscopy-based systems. It may be advantageous to combine modern imaging modalities such as 3D fluoroscopy with computer-assisted navigation for percutaneous screw fixation in the posterior pelvis.

Accuracy of computer-aided diagnosis based on narrow-band imaging endocytoscopy for diagnosing colorectal lesions: comparison with experts

Abstract: Real-time characterization of colorectal lesions during colonoscopy is important for reducing medical costs, given that the need for a pathological diagnosis can be omitted if the accuracy of the diagnostic modality is sufficiently high. However, it is sometimes difficult for community-based gastroenterologists to achieve the required level of diagnostic accuracy. In this regard, we developed a computer-aided diagnosis (CAD) system based on endocytoscopy (EC) to evaluate cellular, glandular, and vessel structure atypia in vivo. The purpose of this study was to compare the diagnostic ability and efficacy of this CAD system with the performances of human expert and trainee endoscopists. We developed a CAD system based on EC with narrow-band imaging that allowed microvascular evaluation without dye (ECV-CAD). The CAD algorithm was programmed based on texture analysis and provided a two-class diagnosis of neoplastic or non-neoplastic, with probabilities. We validated the diagnostic ability of the ECV-CAD system using 173 randomly selected EC images (49 non-neoplasms, 124 neoplasms). The images were evaluated by the CAD and by four expert endoscopists and three trainees. The diagnostic accuracies for distinguishing between neoplasms and non-neoplasms were calculated. ECV-CAD had higher overall diagnostic accuracy than trainees (87.8 vs 63.4%; $$P=0.01$$ ), but similar to experts (87.8 vs 84.2%; $$P=0.76$$ ). With regard to high-confidence cases, the overall accuracy of ECV-CAD was also higher than trainees (93.5 vs 71.7%; $$P<0.001$$ ) and comparable to experts (93.5 vs 90.8%; $$P=0.38$$ ). ECV-CAD showed better diagnostic accuracy than trainee endoscopists and was comparable to that of experts. ECV-CAD could thus be a powerful decision-making tool for less-experienced endoscopists.

Application fields for the new Object Management Group (OMG) Standards Case Management Model and Notation (CMMN) and Decision Management Notation (DMN) in the perioperative field

Abstract: Medical processes can be modeled using different methods and notations Currently used modeling systems like Business Process Model and Notation (BPMN) are not capable of describing the highly flexible and variable medical processes in sufficient detail We combined two modeling systems, Business Process Management (BPM) and Adaptive Case Management (ACM), to be able to model non-deterministic medical processes We used the new Standards Case Management Model and Notation (CMMN) and Decision Management Notation (DMN) First, we explain how CMMN, DMN and BPMN could be used to model non-deterministic medical processes We applied this methodology to model 79 cataract operations provided by University Hospital Leipzig, Germany, and four cataract operations provided by University Eye Hospital Tuebingen, Germany Our model consists of 85 tasks and about 20 decisions in BPMN We were able to expand the system with more complex situations that might appear during an intervention An effective modeling of the cataract intervention is possible using the combination of BPM and ACM The combination gives the possibility to depict complex processes with complex decisions This combination allows a significant advantage for modeling perioperative processes

Scale-adaptive supervoxel-based random forests for liver tumor segmentation in dynamic contrast-enhanced CT scans

Abstract: Toward an efficient clinical management of hepatocellular carcinoma (HCC), we propose a classification framework dedicated to tumor necrosis rate estimation from dynamic contrast-enhanced CT scans. Based on machine learning, it requires weak interaction efforts to segment healthy, active and necrotic liver tissues. Our contributions are two-fold. First, we apply random forest (RF) on supervoxels using multi-phase supervoxel-based features that discriminate tissues based on their dynamic in response to contrast agent injection. Second, we extend this technique in a hierarchical multi-scale fashion to deal with multiple spatial extents and appearance heterogeneity. It translates in an adaptive data sampling scheme combining RF and hierarchical multi-scale tree resulting from recursive supervoxel decomposition. By concatenating multi-phase features across the hierarchical multi-scale tree to describe leaf supervoxels, we enable RF to automatically infer the most informative scales without defining any explicit rules on how to combine them. Assessment on clinical data confirms the benefits of multi-phase information embedded in a multi-scale supervoxel representation for HCC tumor segmentation. Dedicated but not limited only to HCC management, both contributions reach further steps toward more accurate multi-label tissue classification.

Automatic detection of new tumors and tumor burden evaluation in longitudinal liver CT scan studies

Abstract: Radiological longitudinal follow-up of liver tumors in CT scans is the standard of care for disease progression assessment and for liver tumor therapy. Finding new tumors in the follow-up scan is essential to determine malignancy, to evaluate the total tumor burden, and to determine treatment efficacy. Since new tumors are typically small, they may be missed by examining radiologists. We describe a new method for the automatic detection and segmentation of new tumors in longitudinal liver CT studies and for liver tumors burden quantification. Its inputs are the baseline and follow-up CT scans, the baseline tumors delineation, and a tumor appearance prior model. Its outputs are the new tumors segmentations in the follow-up scan, the tumor burden quantification in both scans, and the tumor burden change. Our method is the first comprehensive method that is explicitly designed to find new liver tumors. It integrates information from the scans, the baseline known tumors delineations, and a tumor appearance prior model in the form of a global convolutional neural network classifier. Unlike other deep learning-based methods, it does not require large tagged training sets. Our experimental results on 246 tumors, of which 97 were new tumors, from 37 longitudinal liver CT studies with radiologist approved ground-truth segmentations, yields a true positive new tumors detection rate of 86 versus 72% with stand-alone detection, and a tumor burden volume overlap error of 16%. New tumors detection and tumor burden volumetry are important for diagnosis and treatment. Our new method enables a simplified radiologist-friendly workflow that is potentially more accurate and reliable than the existing one by automatically and accurately following known tumors and detecting new tumors in the follow-up scan.

Three-dimensional texture features from intensity and high-order derivative maps for the discrimination between bladder tumors and wall tissues via MRI

Abstract: This study aims to determine the three-dimensional (3D) texture features extracted from intensity and high-order derivative maps that could reflect textural differences between bladder tumors and wall tissues, and propose a noninvasive, image-based strategy for bladder tumor differentiation preoperatively. A total of 62 cancerous and 62 wall volumes of interest (VOI) were extracted from T2-weighted MRI datasets of 62 patients with pathologically confirmed bladder cancer. To better reflect heterogeneous distribution of tumor tissues, 3D high-order derivative maps (the gradient and curvature maps) were calculated from each VOI. Then 3D Haralick features based on intensity and high-order derivative maps and Tamura features based on intensity maps were extracted from each VOI. Statistical analysis and recursive feature elimination-based support vector machine classifier (RFE-SVM) was proposed to first select the features with significant differences and then obtain a more predictive and compact feature subset to verify its differentiation performance. From each VOI, a total of 58 texture features were derived. Among them, 37 features showed significant inter-class differences ( $$P\le 0.01$$ ). With 29 optimal features selected by RFE-SVM, the classification results namely the sensitivity, specificity, accuracy and area under the curve (AUC) of the receiver operating characteristics were 0.9032, 0.8548, 0.8790 and 0.9045, respectively. By using synthetic minority oversampling technique to augment the sample number of each group to 200, the sensitivity, specificity, accuracy an AUC value of the feature selection-based classification were improved to 0.8967, 0.8780, 0.8874 and 0.9416, respectively. Our results suggest that 3D texture features derived from intensity and high-order derivative maps can better reflect heterogeneous distribution of cancerous tissues. Texture features optimally selected together with sample augmentation could improve the performance on differentiating bladder carcinomas from wall tissues, suggesting a potential way for tumor noninvasive staging of bladder cancer preoperatively.

Design, development and clinical validation of computer-aided surgical simulation system for streamlined orthognathic surgical planning

Abstract: There are many proven problems associated with traditional surgical planning methods for orthognathic surgery. To address these problems, we developed a computer-aided surgical simulation (CASS) system, the AnatomicAligner, to plan orthognathic surgery following our streamlined clinical protocol. The system includes six modules: image segmentation and three-dimensional (3D) reconstruction, registration and reorientation of models to neutral head posture, 3D cephalometric analysis, virtual osteotomy, surgical simulation, and surgical splint generation. The accuracy of the system was validated in a stepwise fashion: first to evaluate the accuracy of AnatomicAligner using 30 sets of patient data, then to evaluate the fitting of splints generated by AnatomicAligner using 10 sets of patient data. The industrial gold standard system, Mimics, was used as the reference. When comparing the results of segmentation, virtual osteotomy and transformation achieved with AnatomicAligner to the ones achieved with Mimics, the absolute deviation between the two systems was clinically insignificant. The average surface deviation between the two models after 3D model reconstruction in AnatomicAligner and Mimics was 0.3 mm with a standard deviation (SD) of 0.03 mm. All the average surface deviations between the two models after virtual osteotomy and transformations were smaller than 0.01 mm with a SD of 0.01 mm. In addition, the fitting of splints generated by AnatomicAligner was at least as good as the ones generated by Mimics. We successfully developed a CASS system, the AnatomicAligner, for planning orthognathic surgery following the streamlined planning protocol. The system has been proven accurate. AnatomicAligner will soon be available freely to the boarder clinical and research communities.

Computer-aided cephalometric landmark annotation for CBCT data

Abstract: Nowadays, with the increased diffusion of Cone Beam Computerized Tomography (CBCT) scanners in dental and maxillo-facial practice, 3D cephalometric analysis is emerging. Maxillofacial surgeons and dentists make wide use of cephalometric analysis in diagnosis, surgery and treatment planning. Accuracy and repeatability of the manual approach, the most common approach in clinical practice, are limited by intra- and inter-subject variability in landmark identification. So, we propose a computer-aided landmark annotation approach that estimates the three-dimensional (3D) positions of 21 selected landmarks. The procedure involves an adaptive cluster-based segmentation of bone tissues followed by an intensity-based registration of an annotated reference volume onto a patient Cone Beam CT (CBCT) head volume. The outcomes of the annotation process are presented to the clinician as a 3D surface of the patient skull with the estimate landmark displayed on it. Moreover, each landmark is centered into a spherical confidence region that can help the clinician in a subsequent manual refinement of the annotation. The algorithm was validated onto 18 CBCT images. Automatic segmentation shows a high accuracy level with no significant difference between automatically and manually determined threshold values. The overall median value of the localization error was equal to 1.99 mm with an interquartile range (IQR) of 1.22–2.89 mm. The obtained results are promising, segmentation was proved to be very robust and the achieved accuracy level in landmark annotation was acceptable for most of landmarks and comparable with other available methods.

Detection and grading of prostate cancer using temporal enhanced ultrasound: combining deep neural networks and tissue mimicking simulations

Abstract: Temporal Enhanced Ultrasound (TeUS) has been proposed as a new paradigm for tissue characterization based on a sequence of ultrasound radio frequency (RF) data. We previously used TeUS to successfully address the problem of prostate cancer detection in the fusion biopsies. In this paper, we use TeUS to address the problem of grading prostate cancer in a clinical study of 197 biopsy cores from 132 patients. Our method involves capturing high-level latent features of TeUS with a deep learning approach followed by distribution learning to cluster aggressive cancer in a biopsy core. In this hypothesis-generating study, we utilize deep learning based feature visualization as a means to obtain insight into the physical phenomenon governing the interaction of temporal ultrasound with tissue. Based on the evidence derived from our feature visualization, and the structure of tissue from digital pathology, we build a simulation framework for studying the physical phenomenon underlying TeUS-based tissue characterization. Results from simulation and feature visualization corroborated with the hypothesis that micro-vibrations of tissue microstructure, captured by low-frequency spectral features of TeUS, can be used for detection of prostate cancer.

The impact of manual threshold selection in medical additive manufacturing

Abstract: Medical additive manufacturing requires standard tessellation language (STL) models. Such models are commonly derived from computed tomography (CT) images using thresholding. Threshold selection can be performed manually or automatically. The aim of this study was to assess the impact of manual and default threshold selection on the reliability and accuracy of skull STL models using different CT technologies. One female and one male human cadaver head were imaged using multi-detector row CT, dual-energy CT, and two cone-beam CT scanners. Four medical engineers manually thresholded the bony structures on all CT images. The lowest and highest selected mean threshold values and the default threshold value were used to generate skull STL models. Geometric variations between all manually thresholded STL models were calculated. Furthermore, in order to calculate the accuracy of the manually and default thresholded STL models, all STL models were superimposed on an optical scan of the dry female and male skulls (“gold standard”). The intra- and inter-observer variability of the manual threshold selection was good (intra-class correlation coefficients >0.9). All engineers selected grey values closer to soft tissue to compensate for bone voids. Geometric variations between the manually thresholded STL models were 0.13 mm (multi-detector row CT), 0.59 mm (dual-energy CT), and 0.55 mm (cone-beam CT). All STL models demonstrated inaccuracies ranging from −0.8 to +1.1 mm (multi-detector row CT), −0.7 to +2.0 mm (dual-energy CT), and −2.3 to +4.8 mm (cone-beam CT). This study demonstrates that manual threshold selection results in better STL models than default thresholding. The use of dual-energy CT and cone-beam CT technology in its present form does not deliver reliable or accurate STL models for medical additive manufacturing. New approaches are required that are based on pattern recognition and machine learning algorithms.

Force-assisted ultrasound imaging system through dual force sensing and admittance robot control.

Abstract: Ultrasound imaging has been a gold standard for clinical diagnoses due to its unique advantages compared to other imaging modalities including: low cost, noninvasiveness, and safeness to the human body. However, the ultrasound scanning process requires applying a large force over extended periods of time, often in uncomfortable postures in order to maintain the desired orientation. This physical requirement over sonographers’ careers often leads to musculoskeletal pain and strain injuries. To address this problem, we propose a cooperatively controlled robotic ultrasound system to reduce the force sonographers apply. The proposed system consists of two key components: a six-axis robotic arm that holds and actuates the ultrasound probe, and a dual force sensor setup that enables cooperative control and adaptive force assistance. With the admittance force control, the robotic arm complies with the motion of the operator, while assisting with force during the scanning. We validated the system through a user study involving expert sonographers and lay people and demonstrated 32–73% reduction in human applied force and 8– 18% improvement in image stability. These results indicate that the system has the potential to not only reduce the burden on the sonographer, but also provide more stable ultrasound scanning.

The production of digital and printed resources from multiple modalities using visualization and three-dimensional printing techniques

Abstract: Virtual digital resources and printed models have become indispensable tools for medical training and surgical planning. Nevertheless, printed models of soft tissue organs are still challenging to reproduce. This study adopts open source packages and a low-cost desktop 3D printer to convert multiple modalities of medical images to digital resources (volume rendering images and digital models) and lifelike printed models, which are useful to enhance our understanding of the geometric structure and complex spatial nature of anatomical organs. Neuroimaging technologies such as CT, CTA, MRI, and TOF-MRA collect serial medical images. The procedures for producing digital resources can be divided into volume rendering and medical image reconstruction. To verify the accuracy of reconstruction, this study presents qualitative and quantitative assessments. Subsequently, digital models are archived as stereolithography format files and imported to the bundled software of the 3D printer. The printed models are produced using polylactide filament materials. We have successfully converted multiple modalities of medical images to digital resources and printed models for both hard organs (cranial base and tooth) and soft tissue organs (brain, blood vessels of the brain, the heart chambers and vessel lumen, and pituitary tumor). Multiple digital resources and printed models were provided to illustrate the anatomical relationship between organs and complicated surrounding structures. Three-dimensional printing (3DP) is a powerful tool to produce lifelike and tangible models. We present an available and cost-effective method for producing both digital resources and printed models. The choice of modality in medical images and the processing approach is important when reproducing soft tissue organs models. The accuracy of the printed model is determined by the quality of organ models and 3DP. With the ongoing improvement of printing techniques and the variety of materials available, 3DP will become an indispensable tool in medical training and surgical planning.