Open AccessProceedings Article
Improved training of wasserstein GANs
Ishaan Gulrajani,Faruk Ahmed,Martin Arjovsky,Vincent Dumoulin,Aaron Courville +4 more
- Vol. 30, pp 5769-5779
TLDR
The authors proposed to penalize the norm of the gradient of the critic with respect to its input to improve the training stability of Wasserstein GANs and achieve stable training of a wide variety of GAN architectures with almost no hyperparameter tuning.Abstract:
Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. The recently proposed Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only poor samples or fail to converge. We find that these problems are often due to the use of weight clipping in WGAN to enforce a Lipschitz constraint on the critic, which can lead to undesired behavior. We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models with continuous generators. We also achieve high quality generations on CIFAR-10 and LSUN bedrooms.read more
Citations
More filters
Journal ArticleDOI
A cost effective solution for pavement crack inspection using cameras and deep neural networks
Qipei Mei,Mustafa Gül +1 more
TL;DR: A novel method called ConnCrack combining conditional Wasserstein generative adversarial network and connectivity maps is proposed for road crack detection, which achieves state-of-the-art performance compared with other existing methods in terms of precision, recall and F1 score.
Posted Content
Wasserstein Distance based Deep Adversarial Transfer Learning for Intelligent Fault Diagnosis.
TL;DR: A novel DTL approach to intelligent fault diagnosis, namely Wasserstein Distance based Deep Transfer Learning (WD-DTL), to learn domain feature representations (generated by a CNN based feature extractor) and to minimize the distributions between the source and target domains through adversarial training.
Journal ArticleDOI
Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time.
TL;DR: This paper proposes to learn 3D LUTs from annotated data using pairwise or unpaired learning and learns an image-adaptive for flexible photo enhancement, which outperforms the state-of-the-art photo enhancement methods by a large margin.
Posted Content
A Survey on Generative Adversarial Networks: Variants, Applications, and Training
Abdul Jabbar,Xi Li,Bourahla Omar +2 more
TL;DR: This work surveys several training solutions proposed by different researchers to stabilize GAN training, and surveys the original GAN model and its modified classical versions, and detail analysis of various GAN applications in different domains.
Proceedings ArticleDOI
Synthesizing Diverse Lung Nodules Wherever Massively: 3D Multi-Conditional GAN-Based CT Image Augmentation for Object Detection
Changhee Han,Yoshiro Kitamura,Akira Kudo,Akimichi Ichinose,Leonardo Rundo,Yujiro Furukawa,Kazuki Umemoto,Yuanzhong Li,Hideki Nakayama +8 more
TL;DR: The results show that 3D Convolutional Neural Network-based detection can achieve higher sensitivity under any nodule size/attenuation at fixed False Positive rates and overcome the medical data paucity with the MCGAN-generated realistic nodules–even expert physicians fail to distinguish them from the real ones in Visual Turing Test.
References
More filters
Dissertation
Learning Multiple Layers of Features from Tiny Images
TL;DR: In this paper, the authors describe how to train a multi-layer generative model of natural images, using a dataset of millions of tiny colour images, described in the next section.
Journal ArticleDOI
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
TL;DR: This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units that are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediate-reinforcement tasks and certain limited forms of delayed-reInforcement tasks, and they do this without explicitly computing gradient estimates.
Posted Content
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
TL;DR: This work introduces a class of CNNs called deep convolutional generative adversarial networks (DCGANs), that have certain architectural constraints, and demonstrates that they are a strong candidate for unsupervised learning.
Posted Content
Improved Techniques for Training GANs
TL;DR: In this article, the authors present a variety of new architectural features and training procedures that apply to the generative adversarial networks (GANs) framework and achieve state-of-the-art results in semi-supervised classification on MNIST, CIFAR-10 and SVHN.
Proceedings Article
Categorical Reparameterization with Gumbel-Softmax
Eric Jang,Shixiang Gu,Ben Poole +2 more
TL;DR: Gumbel-Softmax as mentioned in this paper replaces the non-differentiable samples from a categorical distribution with a differentiable sample from a novel Gumbel softmax distribution, which has the essential property that it can be smoothly annealed into the categorical distributions.