Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Neural Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets. In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

/pdf/a-survey-on-generative-adversarial-networks-for-imbalance-3ec2b2ie2d.pdf

A survey on generative adversarial networks for imbalance problems in computer vision tasks

Abstract The study of artificial intelligence (AI) has been a continuous endeavor of scientists and engineers for over 65 years. The simple contention is that human-created machines can do more than just labor-intensive work; they can develop human-like intelligence. Being aware or not, AI has penetrated into our daily lives, playing novel roles in industry, healthcare, transportation, education, and many more areas that are close to the general public. AI is believed to be one of the major drives to change socio-economical lives. In another aspect, AI contributes to the advancement of state-of-the-art technologies in many fields of study, as helpful tools for groundbreaking research. However, the prosperity of AI as we witness today was not established smoothly. During the past decades, AI has struggled through historical stages with several winters. Therefore, at this juncture, to enlighten future development, it is time to discuss the past, present, and have an outlook on AI. In this article, we will discuss from a historical perspective how challenges were faced on the path of revolution of both the AI tools and the AI systems. Especially, in addition to the technical development of AI in the short to mid-term, thoughts and insights are also presented regarding the symbiotic relationship of AI and humans in the long run. 

/pdf/quo-vadis-artificial-intelligence-k7yzl1rc.pdf

Quo vadis artificial intelligence?

The content-based remote sensing image retrieval (CBRSIR) has recently become a hot topic due to its wide applications in analysis of remote sensing data. However, since conventional CBRSIR is unsuitable in harsh environments, this article focuses on the cross-modality CBRSIR (CM-CBRSIR) between synthetic aperture radar (SAR) and optical images. Besides the large interclass and small intraclass in CBRSIR, CM-CBRSIR is limited by prominent modality discrepancy caused by different imaging mechanisms. To address this limitation, this study proposes a deep cross-modality hashing network. First, we transform optical images with three channels into four different types of single-channel images to increase diversity of the training modalities. This helps the network to mainly focus on extracting the contour and texture shared features and makes it less sensitive to color information for images across modalities. Second, we combine any type of randomly selected transformed images and its corresponding SAR or optical images to form image pairs that are fed into the networks. The training strategy, with paired image data, eliminates the large cross-modality variations caused by different modalities. Finally, the triplet loss, in combination with the hash function, helps the modal to extract the discriminative features of images and upgrade the retrieval efficiency. To further evaluate the proposed modality, we construct a SAR-optical dual-modality remote sensing image dataset containing 12 categories. Experimental results demonstrate the superiority of the proposed method with regards to efficiency and generality.

/pdf/a-deep-cross-modality-hashing-network-for-sar-and-optical-313j67n730.pdf

A Deep Cross-Modality Hashing Network for SAR and Optical Remote Sensing Images Retrieval

Many image processing, computer graphics, and computer vision problems can be treated as image-to-image translation tasks. Such translation entails learning to map one visual representation of a given input to another representation. Image-to-image translation with generative adversarial networks (GANs) has been intensively studied and applied to various tasks, such as multimodal image-to-image translation, super-resolution translation, object transfiguration-related translation, etc. However, image-to-image translation techniques suffer from some problems, such as mode collapse, instability, and a lack of diversity. This article provides a comprehensive overview of image-to-image translation based on GAN algorithms and its variants. It also discusses and analyzes current state-of-the-art image-to-image translation techniques that are based on multimodal and multidomain representations. Finally, open issues and future research directions utilizing reinforcement learning and three-dimensional (3D) modal translation are summarized and discussed.

https://www.mdpi.com/2073-8994/12/10/1705/pdf

Deep Generative Adversarial Networks for Image-to-Image Translation: A Review

In pathology labs worldwide, we see an increasing number of tissue samples that need to be assessed without the same increase in the number of pathologists. Computational pathology, where digital s...

A Multiscale Approach for Whole-Slide Image Segmentation of five Tissue Classes in Urothelial Carcinoma Slides:

Facial attributes edit can be seen as an image-to-image translation problem, whose goal is to transfer images from the source domain to the target domain. Specially, facial attributes edit aims at changing some semantic attributes of a given face image while keeping the contents of unrelated area unchanged. The great challenge for this problem lies on the lacking of paired data, i.e. we do not have paired face images that only differ on particular attributes. Moreover, to train a good attributes editing model, there always needs a great amount of train data which labeled by hand. If the train data amount was reduced, then the editing performance would decrease accordingly. Strong intelligent systems should be able to learn knowledge from less data samples (similar idea with few-shot learning). To mitigate this limitation, in this paper, we proposed a Siamese-Network based residual attributes learning model to learn the attributes difference in the high-level latent space. Compared to existing models that perform attributes editing based on an attributes classifier, the proposed deep residual attributes learning model utilized relatively weaker information of attribute differences for face image translation. Sufficient qualitative and quantitative experiments conducted on CelebA dataset proved the effectiveness of our proposed method, moreover, we also adopt the proposed residual attributes learning model in two state-of-the-art models under different data usage percentage to show the effectiveness of the proposed model on boosting attribute editing performance under limited data usage. The experiment results proved that the proposed method can improve data utilization efficiency and thus can boost the editing performance when the train data was limited.

/pdf/resattr-gan-unpaired-deep-residual-attributes-learning-for-ifmusl0bbl.pdf

ResAttr-GAN: Unpaired Deep Residual Attributes Learning for Multi-Domain Face Image Translation

With the success of generative adversarial networks (GANs) on various real-world applications, the controllability and security of GANs have raised more and more concerns from the community. Specifically, understanding the latent space of GANs, i.e., obtaining the completely decoupled latent space, is essential for applications in some secure scenarios. At present, there is no quantitative method to measure the decoupling of latent space, which is not conducive to the development of the community. In this article, we propose two methods to measure the sensitivity of latent dimensions: one is a sequential intervention method, and the other is an optimization-based method that measures the sensitivity in both the value and the direction. With these two methods, the decoupling of latent space can be measured by the sparsity of the sensitivity vector obtained. The effectiveness of the proposed methods has been verified by experiments on the representative GANs. Code will be available at https://github.com/iceli1007/latent-analysis-of .

Interpreting the Latent Space of GANs via Measuring Decoupling

Automatic medical image segmentation has wide applications for disease diagnosing. However, it is much more challenging than natural optical image segmentation due to the high-resolution of medical images and the corresponding huge computation cost. The sliding window is a commonly used technique for whole slide image (WSI) segmentation, however, for these methods based on the sliding window, the main drawback is lacking global contextual information for supervision. In this paper, we propose a dual-inputs attention network (denoted as DA-RefineNet) for WSI segmentation, where both local fine-grained information and global coarse information can be efficiently utilized. Sufficient comparative experiments are conducted to evaluate the effectiveness of the proposed method, the results prove that the proposed method can achieve better performance on WSI segmentation compared to methods relying on single-input.

DA-RefineNet:A Dual Input Whole Slide Image Segmentation Algorithm Based on Attention.

Generative adversarial networks(GANs) is a popular generative model. With the development of the deep network, its application is more and more widely. By now, people think that the training of GANs is a two-person zero-sum game(discriminator and generator). The lack of strong supervision information makes the training very difficult, such as non-convergence, mode collapses, gradient disappearance, and the sensitivity of hyperparameters. As we all know, regularization and normalization are commonly used for stability training. This paper reviews and summarizes the research in the regularization and normalization for GAN. All the methods are classified into six groups: Gradient penalty, Norm normalization and regularization, Jacobian regularization, Layer normalization, Consistency regularization, and Self-supervision.

Regularization And Normalization For Generative Adversarial Networks: A Review

SAR (Synthetic Aperture Radar) image retrieval, of which the crucial problem is how to search and organize SAR images effectively, has long been a hotspot in SAR-related researches with the increasing amount of SAR images. Besides, SAR image retrieval is also more challenging compared with normal optical image retrieval due to its complicated imaging mechanism. Many SAR image retrieval methods use the high-dimensional feature to improve accuracy, which brings a high computational cost and storage cost for large scale search. In this paper, a system based on fly algorithm for SAR image retrieval is presented. The fly algorithm can generate hash codes for images effectively by mimicing the fruit fly olfactory circuit, which can greatly reduce the dimension of feature space. Three computational strategies extracted from the logic of an important sensory function are used in the fly algorithm to improve the performance of computational similarity searches. Moreover, another effective binary quantization method, different from the quantization method in fly algorithm, is adopted to generate binary hash code in SAR image retrieval system. Experiment results on two representative SAR image datasets, MSTAR and OpenSAR show that the new method based-on fly algorithm outperform other state-of-the-art SAR image retrieval methods.

Rentuo Tao

Papers

ResAttr-GAN: Unpaired Deep Residual Attributes Learning for Multi-Domain Face Image Translation

Interpreting the Latent Space of GANs via Measuring Decoupling

DA-RefineNet:A Dual Input Whole Slide Image Segmentation Algorithm Based on Attention.

Regularization And Normalization For Generative Adversarial Networks: A Review

SAR image retrieval based-on fly algorithm