Showing papers by "Tom Vercauteren published in 2019"
TL;DR: A deep learning-based interactive segmentation method to improve the results obtained by an automatic CNN and to reduce user interactions during refinement for higher accuracy, and obtains comparable and even higher accuracy with fewer user interventions and less time compared with traditional interactive methods.
Abstract: Accurate medical image segmentation is essential for diagnosis, surgical planning and many other applications. Convolutional Neural Networks (CNNs) have become the state-of-the-art automatic segmentation methods. However, fully automatic results may still need to be refined to become accurate and robust enough for clinical use. We propose a deep learning-based interactive segmentation method to improve the results obtained by an automatic CNN and to reduce user interactions during refinement for higher accuracy. We use one CNN to obtain an initial automatic segmentation, on which user interactions are added to indicate mis-segmentations. Another CNN takes as input the user interactions with the initial segmentation and gives a refined result. We propose to combine user interactions with CNNs through geodesic distance transforms, and propose a resolution-preserving network that gives a better dense prediction. In addition, we integrate user interactions as hard constraints into a back-propagatable Conditional Random Field. We validated the proposed framework in the context of 2D placenta segmentation from fetal MRI and 3D brain tumor segmentation from FLAIR images. Experimental results show our method achieves a large improvement from automatic CNNs, and obtains comparable and even higher accuracy with fewer user interventions and less time compared with traditional interactive methods.
347 citations
TL;DR: In this article, a test-time augmentation-based aleatoric uncertainty was proposed to analyze the effect of different transformations of the input image on the segmentation output, and the results showed that the proposed test augmentation provides a better uncertainty estimation than calculating the testtime dropout-based model uncertainty alone and helps to reduce overconfident incorrect predictions.
305 citations
TL;DR: It is found that test-time augmentation improves brain tumor segmentation accuracy and that the resulting uncertainty information can indicate potential mis-segmentations and help to improve segmentations accuracy.
Abstract: Automatic segmentation of brain tumors from medical images is important for clinical assessment and treatment planning of brain tumors. Recent years have seen an increasing use of convolutional neural networks (CNNs) for this task, but most of them use either 2D networks with relatively low memory requirement while ignoring 3D context, or 3D networks exploiting 3D features while with large memory consumption. In addition, existing methods rarely provide uncertainty information associated with the segmentation result. We propose a cascade of CNNs to segment brain tumors with hierarchical subregions from multi-modal Magnetic Resonance images (MRI), and introduce a 2.5D network that is a trade-off between memory consumption, model complexity and receptive field. In addition, we employ test-time augmentation to achieve improved segmentation accuracy, which also provides voxel-wise and structure-wise uncertainty information of the segmentation result. Experiments with BraTS 2017 dataset showed that our cascaded framework with 2.5D CNNs was one of the top performing methods (second-rank) for the BraTS challenge. We also validated our method with BraTS 2018 dataset and found that test-time augmentation improves brain tumor segmentation accuracy and that the resulting uncertainty information can indicate potential mis-segmentations and help to improve segmentation accuracy.
158 citations
TL;DR: An overview of the current status of miPAI is presented and the thoughts on future perspectives are presented.
89 citations
TL;DR: A systematic review of surgical in vivo label‐free multispectral and HSI systems that have been assessed intraoperatively in adult patients, published over a 10‐year period to May 2018, shows a small number of studies have demonstrated the capabilities of intraoperative in‐vivo label‐ free HSI but further work is needed to fully integrate it into the current surgical workflow.
Abstract: Multispectral and hyperspectral imaging (HSI) are emerging optical imaging techniques with the potential to transform the way surgery is performed but it is not clear whether current systems are capable of delivering real-time tissue characterisation and surgical guidance. We conducted a systematic review of surgical in vivo label-free multispectral and hyperspectral imaging systems that have been assessed intraoperatively in adult patients, published over a 10-year period to May 2018. We analysed 13 studies including 7 different HSI systems. Current in-vivo HSI systems generate an intraoperative tissue oxygenation map or enable tumour detection. Intraoperative tissue oxygenation measurements may help to predict those patients at risk of postoperative complications and in-vivo intraoperative tissue characterisation may be performed with high specificity and sensitivity. All systems utilised a line-scanning or wavelength-scanning method but the spectral range and number of spectral bands employed varied significantly between studies and according to the system's clinical aim. The time to acquire a hyperspectral cube dataset ranged between 5 and 30 seconds. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of intraoperative in-vivo label-free hyperspectral imaging but further work is needed to fully integrate it into the current surgical workflow.
67 citations
TL;DR: This work aims to determine the long‐term clinical outcomes in MS, and to identify early prognostic features of these outcomes.
Abstract: Objective Clinical outcomes in multiple sclerosis (MS) are highly variable. We aim to determine the long-term clinical outcomes in MS, and to identify early prognostic features of these outcomes. Methods One hundred thirty-two people presenting with a clinically isolated syndrome were prospectively recruited between 1984 and 1987, and followed up clinically and radiologically 1, 5, 10, 14, 20, and now 30 years later. All available notes and magnetic resonance imaging scans were reviewed, and MS was defined according to the 2010 McDonald criteria. Results Clinical outcome data were obtained in 120 participants at 30 years. Eighty were known to have developed MS by 30 years. Expanded Disability Status Scale (EDSS) scores were available in 107 participants, of whom 77 had MS; 32 (42%) remained fully ambulatory (EDSS scores ≤3.5), all of whom had relapsing-remitting MS (RRMS), 3 (4%) had RRMS and EDSS scores >3.5, 26 (34%) had secondary progressive MS (all had EDSS scores >3.5), and MS contributed to death in 16 (20%). Of those with MS, 11 received disease-modifying therapy. The strongest early predictors (within 5 years of presentation) of secondary progressive MS at 30 years were presence of baseline infratentorial lesions and deep white matter lesions at 1 year. Interpretation Thirty years after onset, in a largely untreated cohort, there was a divergence of MS outcomes; some people accrued substantial disability early on, whereas others ran a more favorable long-term course. These outcomes could, in part, be predicted by radiological findings from within 1 year of first presentation. ANN NEUROL 2020;87:63-74.
62 citations
TL;DR: This novel and proof-of-concept application of computer-aided endoscopic diagnosis shows that a CNN can accurately classify IPCL patterns as normal or abnormal, and could be used as an in vivo, real-time clinical decision support tool for endoscopists assessing and directing local therapy of ESCN.
Abstract: BackgroundIntrapapillary capillary loops (IPCLs) represent an endoscopically visible feature of early squamous cell neoplasia (ESCN) which correlate with invasion depth – an important factor in the...
60 citations
TL;DR: An unsupervised SR framework based on an adversarial deep neural network with a physically‐inspired cycle consistency, designed to impose some acquisition properties on the super‐resolved images is proposed.
59 citations
13 Oct 2019
TL;DR: Li et al. as discussed by the authors proposed a 2.5D convolutional neural network (CNN) with explicit supervision on the attention maps to enable the CNN to focus on the small target for more accurate segmentation.
Abstract: Automatic segmentation of vestibular schwannoma (VS) tumors from magnetic resonance imaging (MRI) would facilitate efficient and accurate volume measurement to guide patient management and improve clinical workflow. The accuracy and robustness is challenged by low contrast, small target region and low through-plane resolution. We introduce a 2.5D convolutional neural network (CNN) able to exploit the different in-plane and through-plane resolutions encountered in standard of care imaging protocols. We propose an attention module with explicit supervision on the attention maps to enable the CNN to focus on the small target for more accurate segmentation. We also propose a hardness-weighted Dice loss function that gives higher weights to harder voxels to boost the training of CNNs. Experiments with ablation studies on the VS tumor segmentation task show that: (1) our 2.5D CNN outperforms its 2D and 3D counterparts, (2) our supervised attention mechanism outperforms unsupervised attention, (3) the voxel-level hardness-weighted Dice loss improves the segmentation accuracy. Our method achieved an average Dice score and ASSD of 0.87 and 0.43 mm respectively. This will facilitate patient management decisions in clinical practice.
57 citations
TL;DR: A new multiparametric model for placental tissue signal in MRI is proposed and it is shown that the placenta may suffer from several pathologies, which affect this fetal‐maternal exchange.
Abstract: Purpose
The placenta is a vital organ for the exchange of oxygen, nutrients, and waste products between fetus and mother The placenta may suffer from several pathologies, which affect this fetal‐maternal exchange, thus the flow properties of the placenta are of interest in determining the course of pregnancy In this work, we propose a new multiparametric model for placental tissue signal in MRI
Methods
We describe a method that separates fetal and maternal flow characteristics of the placenta using a 3‐compartment model comprising fast and slowly circulating fluid pools, and a tissue pool is fitted to overlapping multiecho T2 relaxometry and diffusion MRI with low b‐values We implemented the combined model and acquisition on a standard 15 Tesla clinical system with acquisition taking less than 20 minutes
Results
We apply this combined acquisition in 6 control singleton placentas Mean myometrial T2 relaxation time was 12363 (±671) ms Mean T2 relaxation time of maternal blood was 20217 (±9298) ms In the placenta, mean T2 relaxation time of the fetal blood component was 14489 (±5442) ms Mean ratio of maternal to fetal blood volume was 116 (±06), and mean fetal blood saturation was 7293 (±2011)% across all 6 cases
Conclusion
The novel acquisition in this work allows the measurement of histologically relevant physical parameters, such as the relative proportions of vascular spaces In the placenta, this may help us to better understand the physiological properties of the tissue in disease
55 citations
TL;DR: In this paper, the authors highlight the crucial need to take context and human factors into account in order to address these challenges, and propose Contextual Artificial Intelligence for Computer-Aided Interventions (CAI4CAI).
Abstract: Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.
13 Oct 2019
TL;DR: In this article, a hetero-modal variational 3D encoder-decoder independently embeds all observed modalities into a shared latent representation, and the missing data and tumour segmentation can be then generated from this embedding.
Abstract: We propose a new deep learning method for tumour segmentation when dealing with missing imaging modalities. Instead of producing one network for each possible subset of observed modalities or using arithmetic operations to combine feature maps, our hetero-modal variational 3D encoder-decoder independently embeds all observed modalities into a shared latent representation. Missing data and tumour segmentation can be then generated from this embedding. In our scenario, the input is a random subset of modalities. We demonstrate that the optimisation problem can be seen as a mixture sampling. In addition to this, we introduce a new network architecture building upon both the 3D U-Net and the Multi-Modal Variational Auto-Encoder (MVAE). Finally, we evaluate our method on BraTS2018 using subsets of the imaging modalities as input. Our model outperforms the current state-of-the-art method for dealing with missing modalities and achieves similar performance to the subset-specific equivalent networks.
German Cancer Research Center1, University of Leicester2, University of Santiago de Compostela3, University of Montpellier4, French Institute of Health and Medical Research5, Ghent University6, Manchester Academic Health Science Centre7, University of Cambridge8, Memorial Sloan Kettering Cancer Center9, Hebron University10, University of Manchester11, Icahn School of Medicine at Mount Sinai12, Katholieke Universiteit Leuven13, Maastricht University14, Heidelberg University15, University of Milan16, Ghent University Hospital17, National Council of Resistance of Iran18, University of Hamburg19, University Hospitals of Leicester NHS Trust20, Telford21, Nottingham University Hospitals NHS Trust22, Coventry Health Care23, Queen Elizabeth Hospital Birmingham24, Blackpool Victoria Hospital25, The Queen's Medical Center26
TL;DR: The comprehensive centralised database and linked biobank is a valuable resource for the radiotherapy community for validating predictive models and biomarkers and will also enable a better understanding of how many people suffer with radiotherapy toxicity.
TL;DR: In this article, a hetero-modal variational 3D encoder-decoder independently embeds all observed modalities into a shared latent representation, and the missing data and tumour segmentation can be then generated from this embedding.
Abstract: We propose a new deep learning method for tumour segmentation when dealing with missing imaging modalities. Instead of producing one network for each possible subset of observed modalities or using arithmetic operations to combine feature maps, our hetero-modal variational 3D encoder-decoder independently embeds all observed modalities into a shared latent representation. Missing data and tumour segmentation can be then generated from this embedding. In our scenario, the input is a random subset of modalities. We demonstrate that the optimisation problem can be seen as a mixture sampling. In addition to this, we introduce a new network architecture building upon both the 3D U-Net and the Multi-Modal Variational Auto-Encoder (MVAE). Finally, we evaluate our method on BraTS2018 using subsets of the imaging modalities as input. Our model outperforms the current state-of-the-art method for dealing with missing modalities and achieves similar performance to the subset-specific equivalent networks.
TL;DR: There is a large degree of variation in vascular density throughout normal term human placentas, and the three-dimensional data created by this technique could be used, with more advanced computer analysis, to further investigate the structure of the vascular tree.
TL;DR: dMRI has the potential to inform the understanding of the microstructural changes that occur within the cranial nerves in various pathologies and new avenues of using dMRI should be explored to optimize and improve its reliability.
Abstract: Objective: This paper presents a systematic review of diffusion MRI (dMRI) and tractography of cranial nerves within the posterior fossa. We assess the effectiveness of the diffusion imaging methods used and examine their clinical applications. Methods: The Pubmed, Web of Science and EMBASE databases were searched from January 1st 1997 to December 11th 2017 to identify relevant publications. Any study reporting the use of diffusion imaging and/or tractography in patients with confirmed cranial nerve pathology was eligible for selection. Study quality was assessed using the Methodological Index for Non-Randomized Studies (MINORS) tool. Results: We included 41 studies comprising 16 studies of patients with trigeminal neuralgia (TN), 22 studies of patients with a posterior fossa tumor and three studies of patients with other pathologies. Most acquisition protocols used single-shot echo planar imaging (88%) with a single b-value of 1,000 s/mm2 (78%) but there was significant variation in the number of gradient directions, in-plane resolution, and slice thickness between studies. dMRI of the trigeminal nerve generated interpretable data in all cases. Analysis of diffusivity measurements found significantly lower fractional anisotropy (FA) values within the root entry zone of nerves affected by TN and FA values were significantly lower in patients with multiple sclerosis. Diffusivity values within the trigeminal nerve correlate with the effectiveness of surgical treatment and there is some evidence that pre-operative measurements may be predictive of treatment outcome. Fiber tractography was performed in 30 studies (73%). Most studies evaluating fiber tractography involved patients with a vestibular schwannoma (82%) and focused on generating tractography of the facial nerve to assist with surgical planning. Deterministic tractography using diffusion tensor imaging was performed in 93% of cases but the reported success rate and accuracy of generating fiber tracts from the acquired diffusion data varied considerably. Conclusions: dMRI has the potential to inform our understanding of the microstructural changes that occur within the cranial nerves in various pathologies. Cranial nerve tractography is a promising technique but new avenues of using dMRI should be explored to optimize and improve its reliability.
TL;DR: In fetuses with open spinal dysraphism, brain stem measurements varied substantially between observers, however, measurements characterizing the posterior fossa could be reliably assessed and were significantly different from normal following a fetal operation.
Abstract: BACKGROUND AND PURPOSE: Fetal MR imaging is part of the comprehensive prenatal assessment of fetuses with open spinal dysraphism. We aimed to assess the reliability of brain stem and posterior fossa measurements; use the reliable measurements to characterize fetuses with open spinal dysraphism versus what can be observed in healthy age-matched controls; and document changes in those within 1 week after prenatal repair. MATERIALS AND METHODS: Retrospective evaluation of 349 MR imaging examinations took place, including 274 in controls and 52 in fetuses with open spinal dysraphism, of whom 23 underwent prenatal repair and had additional early postoperative MR images. We evaluated measurements of the brain stem and the posterior fossa and the ventricular width in all populations for their reliability and differences between the groups. RESULTS: The transverse cerebellar diameter, cerebellar herniation level, clivus-supraocciput angle, transverse diameter of the posterior fossa, posterior fossa area, and ventricular width showed an acceptable intra- and interobserver reliability (intraclass correlation coefficient > 0.5). In fetuses with open spinal dysraphism, these measurements were significantly different from those of healthy fetuses (all with P CONCLUSIONS: In fetuses with open spinal dysraphism, brain stem measurements varied substantially between observers. However, measurements characterizing the posterior fossa could be reliably assessed and were significantly different from normal. Following a fetal operation, these deviations from normal values changed significantly within 1 week.
TL;DR: This work introduces a 2.5D convolutional neural network able to exploit the different in-plane and through-plane resolutions encountered in standard of care imaging protocols and proposes a hardness-weighted Dice loss function that gives higher weights to harder voxels to boost the training of CNNs.
Abstract: Automatic segmentation of vestibular schwannoma (VS) tumors from magnetic resonance imaging (MRI) would facilitate efficient and accurate volume measurement to guide patient management and improve clinical workflow. The accuracy and robustness is challenged by low contrast, small target region and low through-plane resolution. We introduce a 2.5D convolutional neural network (CNN) able to exploit the different in-plane and through-plane resolutions encountered in standard of care imaging protocols. We use an attention module to enable the CNN to focus on the small target and propose a supervision on the learning of attention maps for more accurate segmentation. Additionally, we propose a hardness-weighted Dice loss function that gives higher weights to harder voxels to boost the training of CNNs. Experiments with ablation studies on the VS tumor segmentation task show that: 1) the proposed 2.5D CNN outperforms its 2D and 3D counterparts, 2) our supervised attention mechanism outperforms unsupervised attention, 3) the voxel-level hardness-weighted Dice loss can improve the performance of CNNs. Our method achieved an average Dice score and ASSD of 0.87 and 0.43~mm respectively. This will facilitate patient management decisions in clinical practice.
TL;DR: This model of preterm birth, in the absence of any other contributory events, resulted in measurable neurobehavioral deficits with associated brain structural and Magnetic Resonance Diffusion Tensor Imaging findings.
Abstract: Preterm birth is the most significant problem in contemporary obstetrics accounting for 5–18% of worldwide deliveries. Encephalopathy of prematurity encompasses the multifaceted diffuse brain injury resulting from preterm birth. Current animal models exploring the underlying pathophysiology of encephalopathy of prematurity employ significant insults to generate gross central nervous system abnormalities. To date the exclusive effect of prematurity was only studied in a non-human primate model. Therefore, we aimed to develop a representative encephalopathy of prematurity small animal model only dependent on preterm birth. Time mated New-Zealand white rabbit does were either delivered on 28 (pre-term) or 31 (term) postconceptional days by caesarean section. Neonatal rabbits underwent neurobehavioral evaluation on 32 days post conception and then were transcardially perfuse fixed. Neuropathological assessments for neuron and oligodendrocyte quantification, astrogliosis, apoptosis and cellular proliferation were performed. Lastly, ex-vivo high-resolution Magnetic Resonance Imaging was used to calculate T1 volumetric and Diffusion Tensor Imaging derived fractional anisotropy and mean diffusivity. Preterm birth was associated with a motoric (posture instability, abnormal gait and decreased locomotion) and partial sensory (less pain responsiveness and failing righting reflex) deficits that coincided with global lower neuron densities, less oligodendrocyte precursors, increased apoptosis and less proliferation. These region-specific histological changes corresponded with Magnetic Resonance Diffusion Tensor Imaging differences. The most significant differences were seen in the hippocampus, caudate nucleus and thalamus of the preterm rabbits. In conclusion this model of preterm birth, in the absence of any other contributory events, resulted in measurable neurobehavioral deficits with associated brain structural and Magnetic Resonance Diffusion Tensor Imaging findings.
TL;DR: Some promising results are provided suggesting that haptics may offer a distinct advantage in complex robotic interventions were fragile tissue is manipulated.
Abstract: Robotic minimal invasive surgery is gaining acceptance in surgical care. In contrast with the appreciated three-dimensional vision and enhanced dexterity, haptic feedback is not offered. For this reason, robotics is not considered beneficial for delicate interventions such as the endometriosis. Overall, haptic feedback remains debatable and yet unproven except for some simple scenarios such as fundamentals of laparoscopic surgery exercises. Objective: This work investigates the benefits of haptic feedback on more complex surgical gestures, manipulating delicate tissue through coordination between multiple instruments. Methods: A new training exercise, “endometriosis surgery exercise” (ESE) has been devised approximating the setting for monocular robotic endometriosis treatment. A bimanual bilateral teleoperation setup was designed for laparoscopic laser surgery. Haptic guidance and haptic feedback are, respectively, offered to the operator. User experiments have been conducted to assess the validity of ESE and examine possible advantages of haptic technology during execution of bimanual surgery. Results: Content and face validity of ESE were established by participating surgeons. Surgeons suggested ESE also as a mean to train lasering skills, and interaction forces on endometriotic tissue were found to be significantly lower when a bilateral controller is used. Collisions between instruments and the environment were less frequent and so were situations marked as potentially dangerous. Conclusion: This study provides some promising results suggesting that haptics may offer a distinct advantage in complex robotic interventions were fragile tissue is manipulated. Significance: Patients need to know whether it should be incorporated. Improved understanding of the value of haptics is important as current commercial surgical robots are widely used but do not offer haptics.
TL;DR: Two pruning strategies facilitating the use of bundle adjustment in a sequential fashion are introduced that efficiently exploits the potential of using an electromagnetic tracking system to avoid unnecessary matching attempts between spatially inconsistent image pairs and an aggregated representation of images that allows decreasing the computational complexity of a globally consistent approach.
Abstract: Twin-to-twin transfusion syndrome is a condition in which identical twins share a certain pattern of vascular connections in the placenta. This leads to an imbalance in the blood flow that, if not treated, may result in a fatal outcome for both twins. To treat this condition, a surgeon explores the placenta with a fetoscope to find and photocoagulate all intertwin vascular connections. However, the reduced field of view of the fetoscope complicates their localization and general overview. A much more effective exploration could be achieved with an online mosaic created at exploration time. Currently, accurate, globally consistent algorithms such as bundle adjustment cannot be used due to their offline nature, while online algorithms lack sufficient accuracy. We introduce two pruning strategies facilitating the use of bundle adjustment in a sequential fashion: (1) a technique that efficiently exploits the potential of using an electromagnetic tracking system to avoid unnecessary matching attempts between spatially inconsistent image pairs, and (2) an aggregated representation of images, which we refer to as superframes, that allows decreasing the computational complexity of a globally consistent approach. Quantitative and qualitative results on synthetic and phantom-based datasets demonstrate a better trade-off between efficiency and accuracy.
13 Oct 2019
TL;DR: This work proposes an alternative paradigm in which the location of corresponding image-specific ROIs, defined in one image, within another image is learnt, which results in replacing image registration by a conditional segmentation algorithm, which can build on typical image segmentation networks and their widely-adopted training strategies.
Abstract: Classical pairwise image registration methods search for a spatial transformation that optimises a numerical measure that indicates how well a pair of moving and fixed images are aligned. Current learning-based registration methods have adopted the same paradigm and typically predict, for any new input image pair, dense correspondences in the form of a dense displacement field or parameters of a spatial transformation model. However, in many applications of registration, the spatial transformation itself is only required to propagate points or regions of interest (ROIs). In such cases, detailed pixel- or voxel-level correspondence within or outside of these ROIs often have little clinical value. In this paper, we propose an alternative paradigm in which the location of corresponding image-specific ROIs, defined in one image, within another image is learnt. This results in replacing image registration by a conditional segmentation algorithm, which can build on typical image segmentation networks and their widely-adopted training strategies. Using the registration of 3D MRI and ultrasound images of the prostate as an example to demonstrate this new approach, we report a median target registration error (TRE) of 2.1 mm between the ground-truth ROIs defined on intraoperative ultrasound images and those propagated from the preoperative MR images. Significantly lower (>34%) TREs were obtained using the proposed conditional segmentation compared with those obtained from a previously-proposed spatial-transformation-predicting registration network trained with the same multiple ROI labels for individual image pairs. We conclude this work by using a quantitative bias-variance analysis to provide one explanation of the observed improvement in registration accuracy.
13 Oct 2019
TL;DR: A new generalized Deep Sequential Mosaicking (DSM) framework for fetoscopic videos captured from different settings such as simulation, phantom, and real environments, which extends an existing deep image-based homography model to sequential data by proposing controlled data augmentation and outlier rejection methods.
Abstract: Twin-to-twin transfusion syndrome treatment requires fetoscopic laser photocoagulation of placental vascular anastomoses to regulate blood flow to both fetuses. Limited field-of-view (FoV) and low visual quality during fetoscopy make it challenging to identify all vascular connections. Mosaicking can align multiple overlapping images to generate an image with increased FoV, however, existing techniques apply poorly to fetoscopy due to the low visual quality, texture paucity, and hence fail in longer sequences due to the drift accumulated over time. Deep learning techniques can facilitate in overcoming these challenges. Therefore, we present a new generalized Deep Sequential Mosaicking (DSM) framework for fetoscopic videos captured from different settings such as simulation, phantom, and real environments. DSM extends an existing deep image-based homography model to sequential data by proposing controlled data augmentation and outlier rejection methods. Unlike existing methods, DSM can handle visual variations due to specular highlights and reflection across adjacent frames, hence reducing the accumulated drift. We perform experimental validation and comparison using 5 diverse fetoscopic videos to demonstrate the robustness of our framework.
TL;DR: Patients may require further evaluation with contrast‐enhanced computed tomography (CT) images, however, CT fails to offer the high tissue‐ductal‐vessel contrast‐to‐noise ratio available on T2‐weighted MR imaging.
Abstract: Purpose Magnetic resonance (MR) cholangiopancreatography (MRCP) is an established specialist method for imaging the upper abdomen and biliary/pancreatic ducts. Due to limitations of either MR image contrast or low through-plane resolution, patients may require further evaluation with contrast-enhanced computed tomography (CT) images. However, CT fails to offer the high tissue-ductal-vessel contrast-to-noise ratio available on T2-weighted MR imaging. Methods MR super-resolution reconstruction (SRR) frameworks have the potential to provide high-resolution visualizations from multiple low through-plane resolution single-shot T2-weighted (SST2W) images as currently used during MRCP studies. Here, we (i) optimize the source image acquisition protocols by establishing the ideal number and orientation of SST2W series for MRCP SRR generation, (ii) optimize post-processing protocols for two motion correction candidate frameworks for MRCP SRR, and (iii) perform an extensive validation of the overall potential of upper abdominal SRR, using four expert readers with subspeciality interest in hepato-pancreatico-biliary imaging. Results Obtained SRRs show demonstrable advantages over traditional SST2W MRCP data in terms of anatomical clarity and subjective radiologists' preference scores for a range of anatomical regions that are especially critical for the management of cancer patients. Conclusions Our results underline the potential of using SRR alongside traditional MRCP data for improved clinical diagnosis.
13 Oct 2019
TL;DR: In this article, a Permutohedral Attention Module (PAM) is proposed to efficiently capture non-local characteristics of the image for 3D vertebra segmentation and labeling.
Abstract: Medical image processing tasks such as segmentation often require capturing non-local information. As organs, bones, and tissues share common characteristics such as intensity, shape, and texture, the contextual information plays a critical role in correctly labeling them. Segmentation and labeling is now typically done with convolutional neural networks (CNNs) but the context of the CNN is limited by the receptive field which itself is limited by memory requirements and other properties. In this paper, we propose a new attention module, that we call Permutohedral Attention Module (PAM), to efficiently capture non-local characteristics of the image. The proposed method is both memory and computationally efficient. We provide a GPU implementation of this module suitable for 3D medical imaging problems. We demonstrate the efficiency and scalability of our module with the challenging task of vertebrae segmentation and labeling where context plays a crucial role because of the very similar appearance of different vertebrae.
20 May 2019
TL;DR: The design and operation of a robotic multimodal endoscope with optical ultrasound and white light stereo camera is presented, along with large area surface visualisations of a placenta phantom using the optical ultrasound sensor.
Abstract: Fetoscopy is a technically challenging surgery, due to the dynamic environment and low diameter endoscopes often resulting in a limited field of view. In this paper, we report on the design and operation of a robotic multimodal endoscope with optical ultrasound and white light stereo camera. The manufacture and control of the endoscope is presented, along with large area (80 mm ×80 mm) surface visualisations of a placenta phantom using the optical ultrasound sensor. The repeatability of the surface visualisations was found to be 0. 446 ± 0.139 mm and 0. 267 ± 0.017 mm for a raster and spiral scan, respectively.
TL;DR: In this paper, a new generalized Deep Sequential Mosaicking (DSM) framework is presented for fetoscopic videos captured from different settings such as simulation, phantom, and real environments.
Abstract: Twin-to-twin transfusion syndrome treatment requires fetoscopic laser photocoagulation of placental vascular anastomoses to regulate blood flow to both fetuses. Limited field-of-view (FoV) and low visual quality during fetoscopy make it challenging to identify all vascular connections. Mosaicking can align multiple overlapping images to generate an image with increased FoV, however, existing techniques apply poorly to fetoscopy due to the low visual quality, texture paucity, and hence fail in longer sequences due to the drift accumulated over time. Deep learning techniques can facilitate in overcoming these challenges. Therefore, we present a new generalized Deep Sequential Mosaicking (DSM) framework for fetoscopic videos captured from different settings such as simulation, phantom, and real environments. DSM extends an existing deep image-based homography model to sequential data by proposing controlled data augmentation and outlier rejection methods. Unlike existing methods, DSM can handle visual variations due to specular highlights and reflection across adjacent frames, hence reducing the accumulated drift. We perform experimental validation and comparison using 5 diverse fetoscopic videos to demonstrate the robustness of our framework.
01 Jan 2019
TL;DR: The design, assembly and quantitative evaluation of an add-on system intended to be placed on a commercialized cable-driven flexible endoscope is proposed, which is lightweight and easily exchangeable thanks to the McKibben muscle actuators embedded in its system.
Abstract: The instruments currently used by surgeons for in utero treatment of the twin-to-twin transfusion syndrome (TTTS) are rigid or semi-rigid. Their poor dexterity makes this surgical intervention risky and the surgeon's work very complex. This paper proposes the design, assembly and quantitative evaluation of an add-on system intended to be placed on a commercialized cable-driven flexible endoscope. The add-on system is lightweight and easily exchangeable thanks to the McKibben muscle actuators embedded in its system. The combination of the flexible endoscope and the new add-on unit results in an easy controllable flexible instrument with great potential use in TTTS treatment, and especially for regions that are hard to reach with conventional instruments. The fetoscope has a precision of 7.4% over its entire bending range and allows to decrease the maximum planar force on the body wall of 6.15% compared to the original endoscope. The add-on control system also allows a more stable and precise actuation of the endoscope flexible tip.
TL;DR: In this paper, a multi-hypothesis deep learning framework was proposed to generate pseudo CT from MR/CT images by minimising a combination of the pixel-wise error between pCT and CT and a proposed metric loss that itself is represented by a convolutional neural network.
Abstract: The ability to synthesise Computed Tomography images - commonly known as pseudo CT, or pCT - from MRI input data is commonly assessed using an intensity-wise similarity, such as an L2-norm between the ground truth CT and the pCT. However, given that the ultimate purpose is often to use the pCT as an attenuation map ($\mu$-map) in Positron Emission Tomography Magnetic Resonance Imaging (PET/MRI), minimising the error between pCT and CT is not necessarily optimal. The main objective should be to predict a pCT that, when used as $\mu$-map, reconstructs a pseudo PET (pPET) which is as close as possible to the gold standard PET. To this end, we propose a novel multi-hypothesis deep learning framework that generates pCTs by minimising a combination of the pixel-wise error between pCT and CT and a proposed metric-loss that itself is represented by a convolutional neural network (CNN) and aims to minimise subsequent PET residuals. The model is trained on a database of 400 paired MR/CT/PET image slices. Quantitative results show that the network generates pCTs that seem less accurate when evaluating the Mean Absolute Error on the pCT (69.68HU) compared to a baseline CNN (66.25HU), but lead to significant improvement in the PET reconstruction - 115a.u. compared to baseline 140a.u.