scispace - formally typeset
Open accessJournal ArticleDOI: 10.1007/S11263-021-01431-5

You Only Look Yourself: Unsupervised and Untrained Single Image Dehazing Neural Network

04 Mar 2021-International Journal of Computer Vision (Springer US)-Vol. 129, Iss: 5, pp 1754-1767
Abstract: In this paper, we study two challenging and less-touched problems in single image dehazing, namely, how to make deep learning achieve image dehazing without training on the ground-truth clean image (unsupervised) and an image collection (untrained). An unsupervised model will avoid the intensive labor of collecting hazy-clean image pairs, and an untrained model is a “real” single image dehazing approach which could remove haze based on the observed hazy image only and no extra images are used. Motivated by the layer disentanglement, we propose a novel method, called you only look yourself (YOLY) which could be one of the first unsupervised and untrained neural networks for image dehazing. In brief, YOLY employs three joint subnetworks to separate the observed hazy image into several latent layers, i.e., scene radiance layer, transmission map layer, and atmospheric light layer. After that, three layers are further composed to the hazy image in a self-supervised manner. Thanks to the unsupervised and untrained characteristics of YOLY, our method bypasses the conventional training paradigm of deep models on hazy-clean pairs or a large scale dataset, thus avoids the labor-intensive data collection and the domain shift issue. Besides, our method also provides an effective learning-based haze transfer solution thanks to its layer disentanglement mechanism. Extensive experiments show the promising performance of our method in image dehazing compared with 14 methods on six databases. The code could be accessed at www.pengxi.me .

... read more

Topics: Deep learning (51%)
Citations
  More

12 results found


Journal ArticleDOI: 10.1109/TIP.2020.3016134
Boyun Li1, Yuanbiao Gou1, Jerry Zitao Liu, Hongyuan Zhu2  +2 moreInstitutions (2)
Abstract: In this article, we study two less-touched challenging problems in single image dehazing neural networks, namely, how to remove haze from a given image in an unsupervised and zero-shot manner To the ends, we propose a novel method based on the idea of layer disentanglement by viewing a hazy image as the entanglement of several “simpler” layers, ie , a hazy-free image layer, transmission map layer, and atmospheric light layer The major advantages of the proposed ZID are two-fold First, it is an unsupervised method that does not use any clean images including hazy-clean pairs as the ground-truth Second, ZID is a “zero-shot” method, which just uses the observed single hazy image to perform learning and inference In other words, it does not follow the conventional paradigm of training deep model on a large scale dataset These two advantages enable our method to avoid the labor-intensive data collection and the domain shift issue of using the synthetic hazy images to address the real-world images Extensive comparisons show the promising performance of our method compared with 15 approaches in the qualitative and quantitive evaluations The source code could be found at http://wwwpengxime

... read more

16 Citations


Open accessProceedings Article
Yuanbiao Gou1, Boyun Li1, Zitao Liu2, Songfan Yang2  +1 moreInstitutions (3)
01 Jan 2020-
Abstract: Multi-scale neural networks have shown effectiveness in image restoration tasks, which are usually designed and integrated in a handcrafted manner. Different from the existing labor-intensive handcrafted architecture design paradigms, we present a novel method, termed as multi-sCaLe nEural ARchitecture sEarch for image Restoration (CLEARER), which is a specifically designed neural architecture search (NAS) for image restoration. Our contributions are twofold. On one hand, we design a multi-scale search space that consists of three task-flexible modules. Namely, 1) Parallel module that connects multi-resolution neural blocks in parallel, while preserving the channels and spatial-resolution in each neural block, 2) Transition module remains the existing multi-resolution features while extending them to a lower resolution, 3) Fusion module integrates multi-resolution features by passing the features of the parallel neural blocks to the current neural blocks. On the other hand, we present novel losses which could 1) balance the tradeoff between the model complexity and performance, which is highly expected to image restoration; and 2) relax the discrete architecture parameters into a continuous distribution which approximates to either 0 or 1. As a result, a differentiable strategy could be employed to search when to fuse or extract multi-resolution features, while the discretization issue faced by the gradient-based NAS could be alleviated. The proposed CLEARER could search a promising architecture in two GPU hours. Extensive experiments show the promising performance of our method comparing with nine image denoising methods and eight image deraining approaches in quantitative and qualitative evaluations. The codes are available at https://github.com/limit-scu.

... read more

Topics: Scale (ratio) (68%), Image restoration (54%)

14 Citations


Open accessJournal ArticleDOI: 10.1371/JOURNAL.PONE.0253214
Shunmin An1, Xixia Huang1, Linling Wang1, Zhangjing Zheng1  +1 moreInstitutions (1)
28 Jun 2021-PLOS ONE
Abstract: In water scenes, where hazy images are subject to multiple scattering and where ideal data sets are difficult to collect, many dehazing methods are not as effective as they could be. Therefore, an unsupervised water scene dehazing network using atmospheric multiple scattering model is proposed. Unlike previous image dehazing methods, our method uses the unsupervised neural network and the atmospheric multiple scattering model and solves the problem of difficult acquisition of ideal datasets and the effect of multiple scattering on the image. In our method, in order to embed the atmospheric multiple scattering model into the unsupervised dehazing network, the unsupervised dehazing network uses four branches to estimate the scene radiation layer, transmission map layer, blur kernel layer and atmospheric light layer, the hazy image is then synthesized from the four output layers, minimizing the input hazy image and the output hazy image, where the output scene radiation layer is the final dehazing image. In addition, we constructed unsupervised loss functions which applicable to image dehazing by prior knowledge, i.e., color attenuation energy loss and dark channel loss. The method has a wide range of applications, with haze being thick and variable in marine, river and lake scenes, the method can be used to assist ship vision for target detection or forward road recognition in hazy conditions. Through extensive experiments on synthetic and real-world images, the proposed method is able to recover the details, structure and texture of the water image better than five advanced dehazing methods.

... read more

Topics: Channel (digital image) (51%)

1 Citations


Open accessJournal ArticleDOI: 10.1007/S00530-021-00865-8
27 Nov 2021-Multimedia Systems
Abstract: Haze and fog are big reasons for road accidents. The haze occurrence in the air lowers the images quality captured by visible camera sensors. Haze brings inconvenience to numerous computer vision applications as it diminishes the scene visibility. Haze removal techniques recuperate the color and scene contrast. These haze removal techniques are extensively utilized in numerous applications like outdoor surveillance, object detection, consumer electronics, etc. Haze removal is commonly performed under the physical degradation model, which requires a solution of an ill-posed inverse issue. Different dehazing algorithms was recently proposed to relieve this difficulty and has acknowledged a great deal of consideration. Dehazing is basically accomplished through four major steps: hazy images acquisition process, estimation process (atmospheric light, transmission map, scattering phenomenon, and visibility or haze level), enhancement process (improved visibility level, reduce haze or noise level), restoration process (restore enhanced image, image reconstruction). This four-step dehazing process makes it possible to provide a step-by-step approach to the complex solution of the ill-posed inverse problem. Our detailed survey and experimental analysis on different dehazing methods that will help readers understand the effectiveness of the individual step of the dehazing process and will facilitate development of advanced dehazing algorithms. The overall objective of this review paper is to explore the various methods for efficiently removing the haze and short comings of the earlier presented techniques used in the revolutionary era of image processing applications.

... read more

Topics: Haze (61%)

Open accessPosted ContentDOI: 10.21203/RS.3.RS-386958/V1
Zhang Jian1, Wanjuan Song2Institutions (2)
03 Aug 2021-
Abstract: Image dehazing has always been a challenging topic in image processing. The development of deep learning methods, especially the generative adversarial networks (GAN), provides a new way for image dehazing. In recent years, many deep learning methods based on GAN have been applied to image dehazing. However, GAN has two problems in image dehazing. Firstly, For haze image, haze not only reduces the quality of the image but also blurs the details of the image. For GAN network, it is difficult for the generator to restore the details of the whole image while removing the haze. Secondly, GAN model is defined as a minimax problem, which weakens the loss function. It is difficult to distinguish whether GAN is making progress in the training process. Therefore, we propose a guided generative adversarial dehazing network (GGADN). Different from other generation adversarial networks, GGADN adds a guided module on the generator. The guided module verifies the network of each layer of the generator. At the same time, the details of the map generated by each layer are strengthened. Network training is based on the pre-trained VGG feature model and L1-regularized gradient prior which is developed by new loss function parameters. From the dehazing results of synthetic images and real images, the proposed method is better than the state-of-the-art dehazing methods.

... read more

Topics: Image processing (53%), Real image (53%)

References
  More

42 results found


Open accessProceedings Article
Diederik P. Kingma1, Jimmy Ba2Institutions (2)
01 Jan 2015-
Abstract: We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.

... read more

Topics: Stochastic optimization (63%), Convex optimization (54%), Rate of convergence (52%) ... read more

78,539 Citations


Open accessProceedings Article
Sergey Ioffe1, Christian Szegedy1Institutions (1)
06 Jul 2015-
Abstract: Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization, and in some cases eliminates the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.82% top-5 test error, exceeding the accuracy of human raters.

... read more

23,723 Citations


Open accessPosted Content
Sergey Ioffe1, Christian Szegedy1Institutions (1)
11 Feb 2015-arXiv: Learning
Abstract: Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.

... read more

17,151 Citations


Open accessProceedings Article
Diederik P. Kingma1, Max Welling1Institutions (1)
01 Jan 2014-
Abstract: How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case. Our contributions is two-fold. First, we show that a reparameterization of the variational lower bound yields a lower bound estimator that can be straightforwardly optimized using standard stochastic gradient methods. Second, we show that for i.i.d. datasets with continuous latent variables per datapoint, posterior inference can be made especially efficient by fitting an approximate inference model (also called a recognition model) to the intractable posterior using the proposed lower bound estimator. Theoretical advantages are reflected in experimental results.

... read more

Topics: Approximate inference (67%), Inference (55%), Estimator (53%) ... read more

14,546 Citations


Proceedings ArticleDOI: 10.1109/CVPR.2008.4587643
Robby T. Tan1Institutions (1)
23 Jun 2008-
Abstract: Bad weather, such as fog and haze, can significantly degrade the visibility of a scene. Optically, this is due to the substantial presence of particles in the atmosphere that absorb and scatter light. In computer vision, the absorption and scattering processes are commonly modeled by a linear combination of the direct attenuation and the airlight. Based on this model, a few methods have been proposed, and most of them require multiple input images of a scene, which have either different degrees of polarization or different atmospheric conditions. This requirement is the main drawback of these methods, since in many situations, it is difficult to be fulfilled. To resolve the problem, we introduce an automated method that only requires a single input image. This method is based on two basic observations: first, images with enhanced visibility (or clear-day images) have more contrast than images plagued by bad weather; second, airlight whose variation mainly depends on the distance of objects to the viewer, tends to be smooth. Relying on these two observations, we develop a cost function in the framework of Markov random fields, which can be efficiently optimized by various techniques, such as graph-cuts or belief propagation. The method does not require the geometrical information of the input image, and is applicable for both color and gray images.

... read more

Topics: Visibility (59%)

1,728 Citations


Performance
Metrics
No. of citations received by the Paper in previous years
YearCitations
202110
20202