“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Deep neural networks (DNNs) are widely investigated in medical image classification to achieve automated support for clinical diagnosis. It is necessary to evaluate the robustness of medical DNN tasks against adversarial attacks, as high-stake decision-making will be made based on the diagnosis. Several previous studies have considered simple adversarial attacks. However, the vulnerability of DNNs to more realistic and higher risk attacks, such as universal adversarial perturbation (UAP), which is a single perturbation that can induce DNN failure in most classification tasks has not been evaluated yet. We focus on three representative DNN-based medical image classification tasks (i.e., skin cancer, referable diabetic retinopathy, and pneumonia classifications) and investigate their vulnerability to the seven model architectures of UAPs. We demonstrate that DNNs are vulnerable to both nontargeted UAPs, which cause a task failure resulting in an input being assigned an incorrect class, and to targeted UAPs, which cause the DNN to classify an input into a specific class. The almost imperceptible UAPs achieved > 80% success rates for nontargeted and targeted attacks. The vulnerability to UAPs depended very little on the model architecture. Moreover, we discovered that adversarial retraining, which is known to be an effective method for adversarial defenses, increased DNNs’ robustness against UAPs in only very few cases. Unlike previous assumptions, the results indicate that DNN-based clinical diagnosis is easier to deceive because of adversarial attacks. Adversaries can cause failed diagnoses at lower costs (e.g., without consideration of data distribution); moreover, they can affect the diagnosis. The effects of adversarial defenses may not be limited. Our findings emphasize that more careful consideration is required in developing DNNs for medical imaging and their practical applications.

/pdf/universal-adversarial-attacks-on-deep-neural-networks-for-4ipeztmrcp.pdf

Universal adversarial attacks on deep neural networks for medical image classification

Generative adversarial networks in medical image segmentation: A review

Deep learning in retinal optical coherence tomography (OCT): A comprehensive survey

The adaptive optics (AO) technique is widely used to compensate for ocular aberrations and improve imaging resolution. However, when affected by intraocular scatter, speckle noise, and other factors, the quality of the retinal image will be degraded. To effectively improve the image quality without increasing the imaging system's complexity, the post-processing method of image deblurring is adopted. In this study, we proposed a conditional adversarial network-based method for directly learning an end-to-end mapping between blurry and restored AO retinal images. The proposed model was validated on synthetically generated AO retinal images and real retinal images. The restoration results of synthetic images were evaluated with the metrics of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), perceptual distance, and error rate of cone counting. Moreover, the blind image quality index (BIQI) was used as the no-reference image quality assessment (NR-IQA) algorithm to evaluate the restoration results on real AO retinal images. The experimental results indicate that the images restored by the proposed method have sharper quality and higher signal-to-noise ratio (SNR) when compared with other state-of-the-art methods, which has great practical significance for clinical research and analysis.

Quality improvement of adaptive optics retinal images using conditional adversarial networks

Lesion detection is a critical component of disease diagnosis, but the manual segmentation of lesions in medical images is time-consuming and experience-demanding. These issues have recently been addressed through deep learning models. However, most of the existing algorithms were developed using supervised training, which requires time-intensive manual labeling and prevents the model from detecting unaware lesions. As such, this study proposes a weakly supervised learning network based on CycleGAN for lesions segmentation in full-width optical coherence tomography (OCT) images. The model was trained to reconstruct underlying normal anatomic structures from abnormal input images, then the lesions can be detected by calculating the difference between the input and output images. A customized network architecture and a multi-scale similarity perceptual reconstruction loss were used to extend the CycleGAN model to transfer between objects exhibiting shape deformations. The proposed technique was validated using an open-source retinal OCT image dataset. Image-level anomaly detection and pixel-level lesion detection results were assessed using area-under-curve (AUC) and the Dice similarity coefficient, producing results of 96.94% and 0.8239, respectively, higher than all comparative methods. The average test time required to generate a single full-width image was 0.039 s, which is shorter than that reported in recent studies. These results indicate that our model can accurately detect and segment retinopathy lesions in real-time, without the need for supervised labeling. And we hope this method will be helpful to accelerate the clinical diagnosis process and reduce the misdiagnosis rate.

Weakly supervised anomaly segmentation in retinal OCT images using an adversarial learning approach

Geometrical analysis of cone photoreceptor cells is important not only for ophthalmic diagnosis, but also for research on eye diseases. In this study, an automated cone photoreceptor cell segmentation and identification method based on morphological processing and watershed algorithm is presented for adaptive optics scanning laser ophthalmoscope images. Our method includes steps for image denoising, rough segmentation, fine segmentation, small region removal, and identification. The effectiveness of the proposed method was confirmed by comparing its results with those obtained manually yielding precision, recall, and F1-score values of 93.6%, 98.0% and 95.8%, respectively. The performance of our method is further verified by processing images with different cone photoreceptor cell densities from healthy retina and an image from an eye with diabetic retinopathy. The experimental results show that our algorithm achieved high accuracy in cone photoreceptor cell segmentation and identification in healthy retinas as well as in retina with diabetic retinopathy.

/pdf/automated-cone-photoreceptor-cell-segmentation-and-fgq2ph58hg.pdf

Automated Cone Photoreceptor Cell Segmentation and Identification in Adaptive Optics Scanning Laser Ophthalmoscope Images Using Morphological Processing and Watershed Algorithm

A deep neural network (DNN) can assist in retinopathy screening by automatically classifying patients into normal and abnormal categories according to optical coherence tomography (OCT) images. Typically, OCT images captured from different devices show heterogeneous appearances because of different scan settings; thus, the DNN model trained from one domain may fail if applied directly to a new domain. As data labels are difficult to acquire, we proposed a generative adversarial network-based domain adaptation model to address the cross-domain OCT images classification task, which can extract invariant and discriminative characteristics shared by different domains without incurring additional labeling cost. A feature generator, a Wasserstein distance estimator, a domain discriminator, and a classifier were included in the model to enforce the extraction of domain invariant representations. We applied the model to OCT images as well as public digit images. Results show that the model can significantly improve the classification accuracy of cross-domain images.

/pdf/domain-adaptation-model-for-retinopathy-detection-from-cross-443u5lxu97.pdf

Domain adaptation model for retinopathy detection from cross-domain OCT images

Fluorescein angiography can provide a map of retinal vascular structure and function, which is commonly used in ophthalmology diagnosis, however, this imaging modality may pose risks of harm to the patients. To help physicians reduce the potential risks of diagnosis, an image translation method is adopted. In this work, we proposed a conditional generative adversarial network(GAN) - based method to directly learn the mapping relationship between structure fundus images and fundus fluorescence angiography images. Moreover, local saliency maps, which define each pixel's importance, are used to define a novel saliency loss in the GAN cost function. This facilitates more accurate learning of small-vessel and fluorescein leakage features.

/pdf/generating-fundus-fluorescence-angiography-images-from-3c35fg5akv.pdf

Shi Guohua

Papers

Quality improvement of adaptive optics retinal images using conditional adversarial networks

Weakly supervised anomaly segmentation in retinal OCT images using an adversarial learning approach

Automated Cone Photoreceptor Cell Segmentation and Identification in Adaptive Optics Scanning Laser Ophthalmoscope Images Using Morphological Processing and Watershed Algorithm

Domain adaptation model for retinopathy detection from cross-domain OCT images

Generating Fundus Fluorescence Angiography Images from Structure Fundus Images Using Generative Adversarial Networks