Generative adversarial network in medical imaging: A review.

Over the past decade, deep convolutional neural networks have been widely adopted for medical image segmentation and shown to achieve adequate performance. However, due to inherent inductive biases present in convolutional architectures, they lack understanding of long-range dependencies in the image. Recently proposed transformer-based architectures that leverage self-attention mechanism encode long-range dependencies and learn representations that are highly expressive. This motivates us to explore transformer-based solutions and study the feasibility of using transformer-based network architectures for medical image segmentation tasks. Majority of existing transformer-based network architectures proposed for vision applications require large-scale datasets to train properly. However, compared to the datasets for vision applications, in medical imaging the number of data samples is relatively low, making it difficult to efficiently train transformers for medical imaging applications. To this end, we propose a gated axial-attention model which extends the existing architectures by introducing an additional control mechanism in the self-attention module. Furthermore, to train the model effectively on medical images, we propose a Local-Global training strategy (LoGo) which further improves the performance. Specifically, we operate on the whole image and patches to learn global and local features, respectively. The proposed Medical Transformer (MedT) is evaluated on three different medical image segmentation datasets and it is shown that it achieves better performance than the convolutional and other related transformer-based architectures. Code: https://github.com/jeya-maria-jose/Medical-Transformer

Medical Transformer: Gated Axial-Attention for Medical Image Segmentation

Deep Learning in Medical Ultrasound Analysis: A Review

https://fisica.cab.cnea.gov.ar/gpgpu/images/charlas/medical_image_processing_on_the_gpu.pdf

Medical Image Processing on the GPU : Past, Present and Future

Transformer, which can benefit from global (long-range) information modeling using self-attention mechanisms, has been successful in natural language processing and 2D image classification recently. However, both local and global features are crucial for dense prediction tasks, especially for 3D medical image segmentation. In this paper, we for the first time exploit Transformer in 3D CNN for MRI Brain Tumor Segmentation and propose a novel network named TransBTS based on the encoder-decoder structure. To capture the local 3D context information, the encoder first utilizes 3D CNN to extract the volumetric spatial feature maps. Meanwhile, the feature maps are reformed elaborately for tokens that are fed into Transformer for global feature modeling. The decoder leverages the features embedded by Transformer and performs progressive upsampling to predict the detailed segmentation map. Extensive experimental results on both BraTS 2019 and 2020 datasets show that TransBTS achieves comparable or higher results than previous state-of-the-art 3D methods for brain tumor segmentation on 3D MRI scans. The source code is available at https://github.com/Wenxuan-1119/TransBTS.

TransBTS: Multimodal Brain Tumor Segmentation Using Transformer.

Current practice in orthopedic surgery relies on intraoperative fluoroscopy as the main imaging modality for localization and visualization of bone tissue, fractures, implants and surgical tool positions. Ultrasound (US) has recently emerged as a potential nonionizing imaging alternative that promises safer operation while remaining relatively cheap and widely available. US images, however, often depict bone structures poorly, making automatic, accurate and robust localization of bone surfaces quite challenging. In this paper, we present a novel technique for automatic bone surface localization in US that uses local phase image information to derive symmetry-based features corresponding to tissue/bone interfaces through the use of 2-D Log-Gabor filters. We validate the performance of the proposed approach quantitatively using realistic phantom and in vitro experiments as well as qualitatively on in vivo data. Results demonstrate that the proposed technique detects bone surfaces with a localization mean error below 0.40 mm. Furthermore, small gaps between bone fragments can be detected with fracture displacement mean error below 0.33 mm for vertical misalignments, and 0.47 mm for horizontal misalignments.

/pdf/bone-surface-localization-in-ultrasound-using-image-phase-2dwzyc08jm.pdf

Bone surface localization in ultrasound using image phase-based features.

Automatic and accurate Gleason grading of histopathology tissue slides is crucial for prostate cancer diagnosis, treatment, and prognosis. Usually, histopathology tissue slides from different institutions show heterogeneous appearances because of different tissue preparation and staining procedures, thus the predictable model learned from one domain may not be applicable to a new domain directly. Here we propose to adopt unsupervised domain adaptation to transfer the discriminative knowledge obtained from the source domain to the target domain without requiring labeling of images at the target domain. The adaptation is achieved through adversarial training to find an invariant feature space along with the proposed Siamese architecture on the target domain to add a regularization that is appropriate for the whole-slide images. We validate the method on two prostate cancer datasets and obtain significant classification improvement of Gleason scores as compared with the baseline models.

/pdf/adversarial-domain-adaptation-for-classification-of-prostate-2cpm4fa8ad.pdf

Adversarial Domain Adaptation for Classification of Prostate Histopathology Whole-Slide Images

Most methods for medical image segmentation use U-Net or its variants as they have been successful in most of the applications. After a detailed analysis of these "traditional" encoder-decoder based approaches, we observed that they perform poorly in detecting smaller structures and are unable to segment boundary regions precisely. This issue can be attributed to the increase in receptive field size as we go deeper into the encoder. The extra focus on learning high level features causes the U-Net based approaches to learn less information about low-level features which are crucial for detecting small structures. To overcome this issue, we propose using an overcomplete convolutional architecture where we project our input image into a higher dimension such that we constrain the receptive field from increasing in the deep layers of the network. We design a new architecture for image segmentation- KiU-Net which has two branches: (1) an overcomplete convolutional network Kite-Net which learns to capture fine details and accurate edges of the input, and (2) U-Net which learns high level features. Furthermore, we also propose KiU-Net 3D which is a 3D convolutional architecture for volumetric segmentation. We perform a detailed study of KiU-Net by performing experiments on five different datasets covering various image modalities like ultrasound (US), magnetic resonance imaging (MRI), computed tomography (CT), microscopic and fundus images. The proposed method achieves a better performance as compared to all the recent methods with an additional benefit of fewer parameters and faster convergence. Additionally, we also demonstrate that the extensions of KiU-Net based on residual blocks and dense blocks result in further performance improvements. The implementation of KiU-Net can be found here: this https URL

KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentation

3D ultrasound (US) is increasingly considered as a viable alternative imaging modality in computer-assisted orthopaedic surgery (CAOS) applications. Automatic bone segmentation from US images, however, remains a challenge due to speckle noise and various other artifacts inherent to US. In this paper, we present intensity invariant three dimensional (3D) local image phase features, obtained using 3D Log-Gabor filter banks, for extracting ridge-like features similar to those that occur at soft tissue/bone interfaces. Our contributions include the novel extension of 2D phase symmetry features to 3D and their use in automatic extraction of bone surfaces and fractured fragments in 3D US. We validate our technique using phantom, in vitro,and in vivoexperiments. Qualitative and quantitative results demonstrate remarkably clear segmentations results of bone surfaces with a localization accuracy of better than 0.62mm and mean errors in estimating fracture displacements below 0.65mm, which will likely be of strong clinical utility.

/pdf/bone-segmentation-and-fracture-detection-in-ultrasound-using-2tigctb3lq.pdf

Ilker Hacihaliloglu

Papers

Medical Transformer: Gated Axial-Attention for Medical Image Segmentation

Bone surface localization in ultrasound using image phase-based features.

Adversarial Domain Adaptation for Classification of Prostate Histopathology Whole-Slide Images

KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentation

Bone Segmentation and Fracture Detection in Ultrasound Using 3D Local Phase Features