“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

A morphable model for the synthesis of 3D faces

This article presents a novel extended reality (XR) and deep-learning-based Internet-of-Medical-Things (IoMT) solution for the COVID-19 telemedicine diagnostic, which systematically combines virtual reality/augmented reality (AR) remote surgical plan/rehearse hardware, customized 5G cloud computing and deep learning algorithms to provide real-time COVID-19 treatment scheme clues. Compared to existing perception therapy techniques, our new technique can significantly improve performance and security. The system collected 25 clinic data from the 347 positive and 2270 negative COVID-19 patients in the Red Zone by 5G transmission. After that, a novel auxiliary classifier generative adversarial network-based intelligent prediction algorithm is conducted to train the new COVID-19 prediction model. Furthermore, The Copycat network is employed for the model stealing and attack for the IoMT to improve the security performance. To simplify the user interface and achieve an excellent user experience, we combined the Red Zone’s guiding images with the Green Zone’s view through the AR navigate clue by using 5G. The XR surgical plan/rehearse framework is designed, including all COVID-19 surgical requisite details that were developed with a real-time response guaranteed. The accuracy, recall, F1-score, and area under the ROC curve (AUC) area of our new IoMT were 0.92, 0.98, 0.95, and 0.98, respectively, which outperforms the existing perception techniques with significantly higher accuracy performance. The model stealing also has excellent performance, with the AUC area of 0.90 in Copycat slightly lower than the original model. This study suggests a new framework in the COVID-19 diagnostic integration and opens the new research about the integration of XR and deep learning for IoMT implementation.

/pdf/trustworthy-and-intelligent-covid-19-diagnostic-iomt-through-25a59fbpm2.pdf

Trustworthy and Intelligent COVID-19 Diagnostic IoMT Through XR and Deep-Learning-Based Clinic Data Access

Ultrasound (US) imaging is widely used for anatomical structure inspection in clinical diagnosis. The training of new sonographers and deep learning based algorithms for US image analysis usually requires a large amount of data. However, obtaining and labeling large-scale US imaging data are not easy tasks, especially for diseases with low incidence. Realistic US image synthesis can alleviate this problem to a great extent. In this paper, we propose a generative adversarial network (GAN) based image synthesis framework. Our main contributions include: (1) we present the first work that can synthesize realistic B-mode US images with high-resolution and customized texture editing features; (2) to enhance structural details of generated images, we propose to introduce auxiliary sketch guidance into a conditional GAN. We superpose the edge sketch onto the object mask and use the composite mask as the network input; (3) to generate high-resolution US images, we adopt a progressive training strategy to gradually generate high-resolution images from low-resolution images. In addition, a feature loss is proposed to minimize the difference of high-level features between the generated and real images, which further improves the quality of generated images; (4) the proposed US image synthesis method is quite universal and can also be generalized to the US images of other anatomical structures besides the three ones tested in our study (lung, hip joint, and ovary); (5) extensive experiments on three large US image datasets are conducted to validate our method. Ablation studies, customized texture editing, user studies, and segmentation tests demonstrate promising results of our method in synthesizing realistic US images.

Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesis

Despite having promising results, style transfer, which requires preparing style images in advance, may result in lack of creativity and accessibility. Following human instruction, on the other hand, is the most natural way to perform artistic style transfer that can significantly improve controllability for visual effect applications. We introduce a new task—language-driven artistic style transfer (LDAST)—to manipulate the style of a content image, guided by a text. We propose contrastive language visual artist (CLVA) that learns to extract visual semantics from style instructions and accomplish LDAST by the patch-wise style discriminator. The discriminator considers the correlation between language and patches of style images or transferred results to jointly embed style instructions. CLVA further compares contrastive pairs of content images and style instructions to improve the mutual relativeness. The results from the same content image can preserve consistent content structures. Besides, they should present analogous style patterns from style instructions that contain similar visual semantics. The experiments show that our CLVA is effective and achieves superb transferred results on LDAST. 

Language-Driven Artistic Style Transfer

One of the most significant challenges in statistical signal processing and machine learning is how to obtain a generative model that can produce samples of large-scale data distribution, such as images and speeches. Generative Adversarial Network (GAN) is an effective method to address this problem. The GANs provide an appropriate way to learn deep representations without widespread use of labeled training data. This approach has attracted the attention of many researchers in computer vision since it can generate a large amount of data without precise modeling of the probability density function (PDF). In GANs, the generative model is estimated via a competitive process where the generator and discriminator networks are trained simultaneously. The generator learns to generate plausible data, and the discriminator learns to distinguish fake data created by the generator from real data samples. Given the rapid growth of GANs over the last few years and their application in various fields, it is necessary to investigate these networks accurately. In this paper, after introducing the main concepts and the theory of GAN, two new deep generative models are compared, the evaluation metrics utilized in the literature and challenges of GANs are also explained. Moreover, the most remarkable GAN architectures are categorized and discussed. Finally, the essential applications in computer vision are examined.

Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developments

Development and improvement of region proposal algorithms have rapidly become one of the most critical research areas over recent years. The perfect accuracy of region-based recognition techniques has led to the use of proposal algorithms as an imperative core in various recognition problems. The main purpose of these algorithms is to extract effective regions of an image with an appropriate number that will reduce the search space and increase detection accuracy. The early development of these algorithms was based on a set of hand-crafted features. Recently, with advances in deep learning techniques, they have been widely and successfully applied to the region proposals. This paper reviews region proposal algorithms, theory, and evaluation metrics and also addresses the existing challenges. In addition, we present a classification for generating proposals, including classical and advanced methods based on hand-crafted features and deep learning, respectively. Both categories are described in details, and an extensive review of recent works is presented. The proposal improvement methods, including ranking algorithms, are also described. In total, more than 60 different algorithms have been studied and classified, and we also point out several applications based on region proposals.

A comprehensive and systematic review on classical and deep learning based region proposal algorithms

Currently, the most prominent object recognition and image labeling techniques are based on the region proposal algorithms. One of the significant challenges of the region proposal algorithms is to achieve high Recall at high overlaps. This paper proposes a new region proposal algorithm using perceptual grouping to generate fitting regions to enhance the Recall at high overlaps. The proposed method comprises segmentation, region merging, based on texture descriptors, and similarity measurement. Furthermore, the algorithm introduces a hybrid approach to compute an efficient threshold. To fully assess the proposed algorithm, well-known metrics such as overlap and Recall are measured. Experimental results are reported on MSRC, VOC2007, VOC2012, and COCO 2017 datasets. The results are compared with segmentation algorithms, and several classical and deep learning-based region proposals. The evaluation results indicate a good improvement of the Recall at high overlaps, such as 0.8 and 0.9, with a reasonable number of regions.

A region proposal algorithm using texture similarity and perceptual grouping

Multiple-query image retrieval is usually utilized in order to enhance performance of the image retrieval system with considering single semantic for a query set. So far, multiple-query image retrieval based on different queries has rarely studied. In this work, we intend to address this problem using a binary component vector. This vector indicates distinct components which exist in an image. The binary component vector is also generated utilizing low-level feature extraction techniques. The final image retrieval process is performed based on this vector. The experimental results show a better performance and less computation in contrary to previous proposed methods.

http://ieeexplore.ieee.org/iel7/7418733/7422296/07422313.pdf

A novel method for multiple-query image retrieval

This paper proposes an efficient algorithm to appropriately generate a limited number of regions, socalled a region proposal algorithm, for resolving computer vision problems. The most important challenge in region proposal technique is to accurately produce a limited number of regions. This literature introduces an algorithm to answer to the challenge through expanding region. The proposed algorithm comprises of two components and in a triple way. The first component divides an image into some non-overlapping regions. Afterwards, each region is developed in adjacent regions based on the 8-connectivity in the second component. In fact, the image is represented by overlapping expanded regions. This component can be executed to three different modes as fixed, all, and efficient-mode. Our proposed algorithm shows better results than existing state-of-the-art segmentation algorithms, including EGBS, Quick shift, and SLIC. The quality of the regions is measured according to the best overlap and recall metrics at MSRC dataset. The results show the good achievement of the overlap and recall as well. The best result is acquired by EGBS and efficient-mode around 24% and 13% improvement for recall and the best overlap, respectively.

Maryam Taghizadeh

Papers

Generative Adversarial Networks (GANs): An Overview of Theoretical Model, Evaluation Metrics, and Recent Developments

A comprehensive and systematic review on classical and deep learning based region proposal algorithms

A region proposal algorithm using texture similarity and perceptual grouping

A novel method for multiple-query image retrieval

Region Expansion Algorithm: A Well-Quality Region Proposal Generation