“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Deep neural networks (DNNs) are widely investigated in medical image classification to achieve automated support for clinical diagnosis. It is necessary to evaluate the robustness of medical DNN tasks against adversarial attacks, as high-stake decision-making will be made based on the diagnosis. Several previous studies have considered simple adversarial attacks. However, the vulnerability of DNNs to more realistic and higher risk attacks, such as universal adversarial perturbation (UAP), which is a single perturbation that can induce DNN failure in most classification tasks has not been evaluated yet. We focus on three representative DNN-based medical image classification tasks (i.e., skin cancer, referable diabetic retinopathy, and pneumonia classifications) and investigate their vulnerability to the seven model architectures of UAPs. We demonstrate that DNNs are vulnerable to both nontargeted UAPs, which cause a task failure resulting in an input being assigned an incorrect class, and to targeted UAPs, which cause the DNN to classify an input into a specific class. The almost imperceptible UAPs achieved > 80% success rates for nontargeted and targeted attacks. The vulnerability to UAPs depended very little on the model architecture. Moreover, we discovered that adversarial retraining, which is known to be an effective method for adversarial defenses, increased DNNs’ robustness against UAPs in only very few cases. Unlike previous assumptions, the results indicate that DNN-based clinical diagnosis is easier to deceive because of adversarial attacks. Adversaries can cause failed diagnoses at lower costs (e.g., without consideration of data distribution); moreover, they can affect the diagnosis. The effects of adversarial defenses may not be limited. Our findings emphasize that more careful consideration is required in developing DNNs for medical imaging and their practical applications.

/pdf/universal-adversarial-attacks-on-deep-neural-networks-for-4ipeztmrcp.pdf

Universal adversarial attacks on deep neural networks for medical image classification

Generative adversarial networks in medical image segmentation: A review

A generative adversarial network (GAN) is one of the most significant research directions in the field of artificial intelligence, and its superior data generation capability has garnered wide attention. In this article, we discuss the recent advancements in GANs, particularly in the medical field. First, the different medical imaging modalities and the principal theory of GANs were analyzed and summarized, after which, the evaluation metrics and training issues were determined. Third, the extension models of GANs were classified and introduced one-by-one. Fourth, the applications of GAN in medical images including cross-modality, augmentation, detection, classification, and reconstruction were illustrated. Finally, the problems we needed to resolve and future directions were discussed. The objective of this review is to provide a comprehensive overview of the GAN, simplify the GAN’s basics, and present the most successful applications in different scenarios.

The Role of Generative Adversarial Network in Medical Image Analysis: An In-depth Survey

Lesion detection is a critical component of disease diagnosis, but the manual segmentation of lesions in medical images is time-consuming and experience-demanding. These issues have recently been addressed through deep learning models. However, most of the existing algorithms were developed using supervised training, which requires time-intensive manual labeling and prevents the model from detecting unaware lesions. As such, this study proposes a weakly supervised learning network based on CycleGAN for lesions segmentation in full-width optical coherence tomography (OCT) images. The model was trained to reconstruct underlying normal anatomic structures from abnormal input images, then the lesions can be detected by calculating the difference between the input and output images. A customized network architecture and a multi-scale similarity perceptual reconstruction loss were used to extend the CycleGAN model to transfer between objects exhibiting shape deformations. The proposed technique was validated using an open-source retinal OCT image dataset. Image-level anomaly detection and pixel-level lesion detection results were assessed using area-under-curve (AUC) and the Dice similarity coefficient, producing results of 96.94% and 0.8239, respectively, higher than all comparative methods. The average test time required to generate a single full-width image was 0.039 s, which is shorter than that reported in recent studies. These results indicate that our model can accurately detect and segment retinopathy lesions in real-time, without the need for supervised labeling. And we hope this method will be helpful to accelerate the clinical diagnosis process and reduce the misdiagnosis rate.

Weakly supervised anomaly segmentation in retinal OCT images using an adversarial learning approach

A deep neural network (DNN) can assist in retinopathy screening by automatically classifying patients into normal and abnormal categories according to optical coherence tomography (OCT) images. Typically, OCT images captured from different devices show heterogeneous appearances because of different scan settings; thus, the DNN model trained from one domain may fail if applied directly to a new domain. As data labels are difficult to acquire, we proposed a generative adversarial network-based domain adaptation model to address the cross-domain OCT images classification task, which can extract invariant and discriminative characteristics shared by different domains without incurring additional labeling cost. A feature generator, a Wasserstein distance estimator, a domain discriminator, and a classifier were included in the model to enforce the extraction of domain invariant representations. We applied the model to OCT images as well as public digit images. Results show that the model can significantly improve the classification accuracy of cross-domain images.

/pdf/domain-adaptation-model-for-retinopathy-detection-from-cross-443u5lxu97.pdf

Domain adaptation model for retinopathy detection from cross-domain OCT images

Fluorescein angiography can provide a map of retinal vascular structure and function, which is commonly used in ophthalmology diagnosis, however, this imaging modality may pose risks of harm to the patients. To help physicians reduce the potential risks of diagnosis, an image translation method is adopted. In this work, we proposed a conditional generative adversarial network(GAN) - based method to directly learn the mapping relationship between structure fundus images and fundus fluorescence angiography images. Moreover, local saliency maps, which define each pixel's importance, are used to define a novel saliency loss in the GAN cost function. This facilitates more accurate learning of small-vessel and fluorescein leakage features.

/pdf/generating-fundus-fluorescence-angiography-images-from-3c35fg5akv.pdf

Generating Fundus Fluorescence Angiography Images from Structure Fundus Images Using Generative Adversarial Networks

A line-scanning ophthalmoscope (LSO) is a retinal imaging technique that has the characteristics of high imaging resolution, wide field of view, and high imaging speed. However, the high-speed imaging with rather short exposure time inevitably reduces the signal intensity, and many factors, such as speckle noise and intraocular scatter, further degrade the signal-to-noise ratio (SNR) of retinal images. To effectively improve the image quality without increasing the LSO system’s complexity, the post-processing method of image super-resolution (SR) is adopted. In this paper, we propose a learning-based multi-frame retinal image SR method that directly learns an end-to-end mapping from low-resolution (LR) image sequences to high-resolution (HR) images. This network was validated on down-sampled and real LSO image sequences. We evaluated the method on a down-sampled dataset with the metrics of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and perceptual distance. Moreover, the power spectra and full width at half maximum (FWHM) were used as the no-reference image quality assessment (NR-IQA) algorithms to evaluate the reconstruction results of the real LSO image sequences. The experimental results indicate that the proposed method can significantly enhance the SNR of LSO images and efficiently improve the resolution of LSO retinal images, which has great practical significance for clinical diagnosis and analysis.

/pdf/enhancement-of-retinal-image-from-line-scanning-4sxz2dcsh9.pdf

Enhancement of Retinal Image From Line-Scanning Ophthalmoscope Using Generative Adversarial Networks

Fundus fluorescein angiography (FA) is an indispensable procedure that can investigate the integrity of retina vasculature. Fluorescein angiograms progress through five phases: pre-arterial, arterial, arteriovenous, venous, and late, and each phase could be an important diagnostic basis for retina-related disease. However, the FA imaging technique may provide risks of harm to the patients. To help physicians reduce the potential risks of diagnosis, we proposed “SequenceGAN”, a novel sequential generative adversarial network that aims to generate FA sequences of critical phases from a structure fundus image. Moreover, a feature-space loss is applied to ensure the generated FA sequences with a better visual effect. The proposed method was qualitatively and quantitatively compared with existing FA image generation methods and image translation methods. The experimental results indicate that the proposed model has better performance on the generation of retina vascular, leakage structures, and characteristics of each angiogram phase, and thus indicates potential value for application in clinical diagnosis.

Wen Kong

Papers

Weakly supervised anomaly segmentation in retinal OCT images using an adversarial learning approach

Domain adaptation model for retinopathy detection from cross-domain OCT images

Generating Fundus Fluorescence Angiography Images from Structure Fundus Images Using Generative Adversarial Networks

Enhancement of Retinal Image From Line-Scanning Ophthalmoscope Using Generative Adversarial Networks

SequenceGAN: Generating Fundus Fluorescence Angiography Sequences from Structure Fundus Image.