Bart R. Thomson

Bio: Bart R. Thomson is an academic researcher from Netherlands Cancer Institute. The author has contributed to research in topics: Image registration. The author has an hindex of 2, co-authored 4 publications receiving 12 citations.

MR-to-US Registration Using Multiclass Segmentation of Hepatic Vasculature with a Reduced 3D U-Net.

Abstract: Accurate hepatic vessel segmentation and registration using ultrasound (US) can contribute to beneficial navigation during hepatic surgery. However, it is challenging due to noise and speckle in US imaging and liver deformation. Therefore, a workflow is developed using a reduced 3D U-Net for segmentation, followed by non-rigid coherent point drift (CPD) registration. By means of electromagnetically tracked US, 61 3D volumes were acquired during surgery. Dice scores of 0.77, 0.65 and 0.66 were achieved for segmentation of all vasculature, hepatic vein and portal vein respectively. This compares to inter-observer variabilities of 0.85, 0.88 and 0.74 respectively. Target registration error at a tumor lesion of interest was lower (7.1 mm) when registration is performed either on the hepatic or the portal vein, compared to using all vasculature (8.9 mm). Using clinical data, we developed a workflow consisting of multi-class segmentation combined with selective non-rigid registration that leads to sufficient accuracy for integration in computer assisted liver surgery.

Hepatic vessel segmentation using a reduced filter 3D U-Net in ultrasound imaging

Abstract: Accurate hepatic vessel segmentation on ultrasound (US) images can be an important tool in the planning and execution of surgery, however proves to be a challenging task due to noise and speckle. Our method comprises a reduced filter 3D U-Net implementation to automatically detect hepatic vasculature in 3D US volumes. A comparison is made between volumes acquired with a 3D probe and stacked 2D US images based on electromagnetic tracking. Experiments are conducted on 67 scans, where 45 are used in training, 12 in validation and 10 in testing. This network architecture yields Dice scores of 0.740 and 0.781 for 3D and stacked 2D volumes respectively, comparing promising to literature and inter-observer performance (Dice = 0.879).

Technical note: Validation of 3D ultrasound for image registration during oncological liver surgery.

Abstract: Purpose: Registration of pre- and intraoperative images is a crucial step of surgical liver navigation, where rigid registration of vessel centerlines is currently commonly used. When using 3D ultrasound (US), accuracy during navigation might be influenced by the size of the intraoperative US volume, yet the relationship between registration accuracy and US volume size is understudied. In this study, we specify an optimal 3D US volume size for registration using varying volumes of liver vasculature. While previous studies measured accuracy at registered fiducials, in this work, accuracy is determined at the target lesion which is clinically the most relevant structure. Methods: Three-dimensional US volumes were acquired in 14 patients after laparotomy and liver mobilization. Manual segmentation of vasculature and centerline extraction was performed. Intraoperative and preoperative vasculature centerlines were registered with coherent point drift, using different sub-volumes (sphere with radius r = 30, 40, …, 120 mm). Accuracy was measured by fiducial registration error (FRE) between vessel centerlines and target registration error (TRE) at the center of the target lesion. Results: The lowest FRE for vessel registration was reached with r = 50 mm (6.5 ± 2.5 mm), the highest with r = 120 mm (7.1 ± 2.1 mm). Clinical accuracy at the target lesion, resulted most accurate (TRE = 8.8 ± 5.0 mm) in sub-volumes with a radius of 50 mm. Smaller US sub-volumes resulted in lower average TREs when compared to larger US sub-volumes (Pearson's correlation coefficient R = 0.91, p < 0.001). Conclusion: Our results indicate that there is a linear correlation between US volume size and registration accuracy at the tumor. Volumes with radii of 50 mm around the target lesion yield higher accuracy (p < 0.05) (Trial number IRBd18032, 11 September 2018).

Hepatic vessel segmentation using a reduced filter 3D U-Net in ultrasound imaging

Abstract: Accurate hepatic vessel segmentation on ultrasound (US) images can be an important tool in the planning and execution of surgery, however proves to be a challenging task due to noise and speckle. Our method comprises a reduced filter 3D U-Net implementation to automatically detect hepatic vasculature in 3D US volumes. A comparison is made between volumes acquired with a 3D probe and stacked 2D US images based on electromagnetic tracking. Experiments are conducted on 67 scans, where 45 are used in training, 12 in validation and 10 in testing. This network architecture yields Dice scores of 0.740 and 0.781 for 3D and stacked 2D volumes respectively, comparing promising to literature and inter-observer performance (Dice = 0.879).

U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications

Abstract: U-net is an image segmentation technique developed primarily for image segmentation tasks. These traits provide U-net with a high utility within the medical imaging community and have resulted in extensive adoption of U-net as the primary tool for segmentation tasks in medical imaging. The success of U-net is evident in its widespread use in nearly all major image modalities, from CT scans and MRI to X-rays and microscopy. Furthermore, while U-net is largely a segmentation tool, there have been instances of the use of U-net in other applications. Given that U-net’s potential is still increasing, this narrative literature review examines the numerous developments and breakthroughs in the U-net architecture and provides observations on recent trends. We also discuss the many innovations that have advanced in deep learning and discuss how these tools facilitate U-net. In addition, we review the different image modalities and application areas that have been enhanced by U-net.

Attention-Guided Deep Neural Network With Multi-Scale Feature Fusion for Liver Vessel Segmentation

Abstract: Liver vessel segmentation is fast becoming a key instrument in the diagnosis and surgical planning of liver diseases. In clinical practice, liver vessels are normally manual annotated by clinicians on each slice of CT images, which is extremely laborious. Several deep learning methods exist for liver vessel segmentation, however, promoting the performance of segmentation remains a major challenge due to the large variations and complex structure of liver vessels. Previous methods mainly using existing UNet architecture, but not all features of the encoder are useful for segmentation and some even cause interferences. To overcome this problem, we propose a novel deep neural network for liver vessel segmentation, called LVSNet, which employs special designs to obtain the accurate structure of the liver vessel. Specifically, we design Attention-Guided Concatenation (AGC) module to adaptively select the useful context features from low-level features guided by high-level features. The proposed AGC module focuses on capturing rich complemented information to obtain more details. In addition, we introduce an innovative multi-scale fusion block by constructing hierarchical residual-like connections within one single residual block, which is of great importance for effectively linking the local blood vessel fragments together. Furthermore, we construct a new dataset containing 40 thin thickness cases (0.625 mm) which consist of CT volumes and annotated vessels. To evaluate the effectiveness of the method with minor vessels, we also propose an automatic stratification method to split major and minor liver vessels. Extensive experimental results demonstrate that the proposed LVSNet outperforms previous methods on liver vessel segmentation datasets. Additionally, we conduct a series of ablation studies that comprehensively support the superiority of the underlying concepts.

Cross-Modal Attention for MRI and Ultrasound Volume Registration

Abstract: Prostate cancer biopsy benefits from accurate fusion of transrectal ultrasound (TRUS) and magnetic resonance (MR) images. In the past few years, convolutional neural networks (CNNs) have been proved powerful in extracting image features crucial for image registration. However, challenging applications and recent advances in computer vision suggest that CNNs are quite limited in its ability to understand spatial correspondence between features, a task in which the self-attention mechanism excels. This paper aims to develop a self-attention mechanism specifically for cross-modal image registration. Our proposed cross-modal attention block effectively maps each of the features in one volume to all features in the corresponding volume. Our experimental results demonstrate that a CNN network designed with the cross-modal attention block embedded outperforms an advanced CNN network 10 times of its size. We also incorporated visualization techniques to improve the interpretability of our network. The source code of our work is available at https://github.com/DIAL-RPI/Attention-Reg.

A novel method to model hepatic vascular network using vessel segmentation, thinning, and completion

Abstract: The accurate modeling of the liver vessel network structure is an important prerequisite for developing a preoperative plan for the liver. Considering that extracting liver blood vessels from patient's abdominal computed tomography(CT) images requires several manual operations, this study proposed an automatic segmentation method of liver vessels based on graph cut, thinning, and vascular combination, which can obtain a complete liver vascular network. First, the CT image was preprocessed by grayscale mapping based on sigmoid function, vessel enhancement based on Hessian filter, and denoising based on anisotropic filter to enhance the grayscale contrast between the vascular and non-vascular parts of the liver. Then, the liver vessels were initially segmented based on the improved three-dimensional graph cut algorithm. Based on the obtained liver vascular structure, the vessel centerline of the liver was then extracted by the proposed thinning algorithm that continuously traversed the foreground voxel points and iteratively deleted the simple points. Finally, the combination of vascular centerline optimization was used to predict and link the vascular centerline fractured portion. The under-segmented liver vessels were complemented based on the complete vascular centerline tree. To verify the proposed hepatic vascular segmentation and complementation algorithm, the open 3D Image Reconstruction for Comparison of Algorithm Database (3Dircadb) was applied to test and quantify the results. The results showed that the proposed algorithm can accurately and effectively segment the vascular network structure from abdominal CT images, and the proposed vascular complementation method can restore the true information of under-segmented liver vessels. Graphical abstract A novel hepatic vessel segmentation method from abdominal CT images was proposed, including graph cut algorithm, centerline extraction, and broken vessel completion. First, the graph cut algorithm was used to obtain the initial segmentation result. Then, the centerline of the initial segmentation result was extracted. Finally, the initial segmentation result was optimized through centerline analysis.

Registration of Untracked 2D Laparoscopic Ultrasound to CT Images of the Liver Using Multi-Labelled Content-Based Image Retrieval

Abstract: Laparoscopic Ultrasound (LUS) is recommended as a standard-of-care when performing laparoscopic liver resections as it images sub-surface structures such as tumours and major vessels. Given that LUS probes are difficult to handle and some tumours are iso-echoic, registration of LUS images to a pre-operative CT has been proposed as an image-guidance method. This registration problem is particularly challenging due to the small field of view of LUS, and usually depends on both a manual initialisation and tracking to compose a volume, hindering clinical translation. In this paper, we extend a proposed registration approach using Content-Based Image Retrieval (CBIR), removing the requirement for tracking or manual initialisation. Pre-operatively, a set of possible LUS planes is simulated from CT and a descriptor generated for each image. Then, a Bayesian framework is employed to estimate the most likely sequence of CT simulations that matches a series of LUS images. We extend our CBIR formulation to use multiple labelled objects and constrain the registration by separating liver vessels into portal vein and hepatic vein branches. The value of this new labeled approach is demonstrated in retrospective data from 5 patients. Results show that, by including a series of 5 untracked images in time, a single LUS image can be registered with accuracies ranging from 5.7 to 16.4 mm with a success rate of 78%. Initialisation of the LUS to CT registration with the proposed framework could potentially enable the clinical translation of these image fusion techniques.