Zero-Shot Image Dehazing
TL;DR: A novel method based on the idea of layer disentanglement by viewing a hazy image as the entanglement of several “simpler” layers, i.e., a haazi-free image layer, transmission map layer, and atmospheric light layer is proposed.
Abstract: In this article, we study two less-touched challenging problems in single image dehazing neural networks, namely, how to remove haze from a given image in an unsupervised and zero-shot manner To the ends, we propose a novel method based on the idea of layer disentanglement by viewing a hazy image as the entanglement of several “simpler” layers, ie , a hazy-free image layer, transmission map layer, and atmospheric light layer The major advantages of the proposed ZID are two-fold First, it is an unsupervised method that does not use any clean images including hazy-clean pairs as the ground-truth Second, ZID is a “zero-shot” method, which just uses the observed single hazy image to perform learning and inference In other words, it does not follow the conventional paradigm of training deep model on a large scale dataset These two advantages enable our method to avoid the labor-intensive data collection and the domain shift issue of using the synthetic hazy images to address the real-world images Extensive comparisons show the promising performance of our method compared with 15 approaches in the qualitative and quantitive evaluations The source code could be found at http://wwwpengxime
Citations
More filters
TL;DR: Zhang et al. as discussed by the authors proposed a Light-DehazeNet (LD-Net) to estimate both the transmission map and the atmospheric light using a transformed atmospheric scattering model, and a color visibility restoration method is proposed to evade the color distortion in the dehaze image.
Abstract: Due to the rapid development of artificial intelligence technology, industrial sectors are revolutionizing in automation, reliability, and robustness, thereby significantly increasing quality and productivity. Most of the surveillance and industrial sectors are monitored by visual sensor networks capturing different surrounding environment images. However, during tempestuous weather conditions, the visual quality of the images is reduced due to contaminated suspended atmospheric particles that affect the overall surveillance systems. To tackle these challenges, this article presents a computationally efficient lightweight convolutional neural network referred to as Light-DehazeNet (LD-Net) for the reconstruction of hazy images. Unlike other learning-based approaches, which separately measure the transmission map and the atmospheric light, our proposed LD-Net jointly estimates both the transmission map and the atmospheric light using a transformed atmospheric scattering model. Furthermore, a color visibility restoration method is proposed to evade the color distortion in the dehaze image. Finally, we conduct extensive experiments using synthetic and natural hazy images. The quantitative and qualitative evaluation on different benchmark hazy datasets verify the superiority of the proposed method over other state-of-the-art image dehazing techniques. Moreover, additional experimentation validates the applicability of the proposed method in the object detection tasks. Considering the lightweight architecture with minimal computational cost, the proposed system is encouraged to be incorporated as an integral part of the vision-based monitoring systems to improve the overall performance.
39 citations
Proceedings Article•
01 Jan 2020TL;DR: A differentiable strategy could be employed to search when to fuse or extract multi-resolution features, while the discretization issue faced by the gradient-based NAS could be alleviated.
Abstract: Multi-scale neural networks have shown effectiveness in image restoration tasks, which are usually designed and integrated in a handcrafted manner. Different from the existing labor-intensive handcrafted architecture design paradigms, we present a novel method, termed as multi-sCaLe nEural ARchitecture sEarch for image Restoration (CLEARER), which is a specifically designed neural architecture search (NAS) for image restoration. Our contributions are twofold. On one hand, we design a multi-scale search space that consists of three task-flexible modules. Namely, 1) Parallel module that connects multi-resolution neural blocks in parallel, while preserving the channels and spatial-resolution in each neural block, 2) Transition module remains the existing multi-resolution features while extending them to a lower resolution, 3) Fusion module integrates multi-resolution features by passing the features of the parallel neural blocks to the current neural blocks. On the other hand, we present novel losses which could 1) balance the tradeoff between the model complexity and performance, which is highly expected to image restoration; and 2) relax the discrete architecture parameters into a continuous distribution which approximates to either 0 or 1. As a result, a differentiable strategy could be employed to search when to fuse or extract multi-resolution features, while the discretization issue faced by the gradient-based NAS could be alleviated. The proposed CLEARER could search a promising architecture in two GPU hours. Extensive experiments show the promising performance of our method comparing with nine image denoising methods and eight image deraining approaches in quantitative and qualitative evaluations. The codes are available at https://github.com/limit-scu.
39 citations
Cites background from "Zero-Shot Image Dehazing"
...As a matter of fact, the advances of image restoration in recent years are benefited from the developments of various handcrafted neural network architectures [27, 14, 15, 19, 24, 40, 31, 34]....
[...]
...To date, a number of image restoration algorithms have been proposed, which achieved remarkable development in numerous practical applications [19, 40, 15, 21]....
[...]
01 Jun 2022
TL;DR: The proposed All-in-one Image Restoration Network (AirNet) consisting of two neural modules, named Contrastive-Based Degraded Encoder (CBDE) and Degradation-Guided Restoration network (DGRN), which outperforms 17 image restoration baselines on four challenging datasets.
Abstract: In this paper, we study a challenging problem in image restoration, namely, how to develop an all-in-one method that could recover images from a variety of unknown corruption types and levels. To this end, we propose an All-in-one Image Restoration Network (AirNet) consisting of two neural modules, named Contrastive-Based Degraded Encoder (CBDE) and Degradation-Guided Restoration Network (DGRN). The major advantages of AirNet are two-fold. First, it is an all-in-one solution which could recover various degraded images in one network. Second, AirNet is free from the prior of the corruption types and levels, which just uses the observed corrupted image to perform inference. These two advantages enable AirNet to enjoy better flexibility and higher economy in real world scenarios wherein the priors on the corruptions are hard to know and the degradation will change with space and time. Extensive experimental results show the proposed method outperforms 17 image restoration baselines on four challenging datasets. The code is available at https://github.com/XLearning-SCU/2022-CVPR-AirNet.
32 citations
01 Jun 2022
TL;DR: This paper proposes a self-augmented image dehazing framework, termed D4 (Dehazing via Decomposing transmission map into Density and Depth) for haze generation and removal, which outperforms state-of-the-art unpaired dehazed methods with much fewer parameters and FLOPs.
Abstract: To overcome the overfitting issue of dehazing models trained on synthetic hazy-clean image pairs, many recent methods attempted to improve models' generalization ability by training on unpaired data. Most of them simply formulate dehazing and rehazing cycles, yet ignore the physical properties of the real-world hazy environment, i.e. the haze varies with density and depth. In this paper, we propose a self-augmented image dehazing framework, termed D4 (Dehazing via Decomposing transmission map into Density and Depth) for haze generation and removal. Instead of merely estimating transmission maps or clean content, the proposed framework focuses on exploring scattering coefficient and depth information contained in hazy and clean images. With estimated scene depth, our method is capable of re-rendering hazy images with different thick-nesses which further benefits the training of the dehazing network. It is worth noting that the whole training process needs only unpaired hazy and clean images, yet succeeded in recovering the scattering coefficient, depth map and clean content from a single hazy image. Comprehensive experiments demonstrate our method outperforms state-of-the-art unpaired dehazing methods with much fewer parameters and FLOPs. Our code is available at https://github.com/YaN9-Y/D4.
26 citations
01 Jun 2022
TL;DR: In this paper , a self-augmented image dehazing framework, termed D4 (Dehazing via Decomposing transmission map into Density and Depth), is proposed for haze generation and removal.
Abstract: To overcome the overfitting issue of dehazing models trained on synthetic hazy-clean image pairs, many recent methods attempted to improve models' generalization ability by training on unpaired data. Most of them simply formulate dehazing and rehazing cycles, yet ignore the physical properties of the real-world hazy environment, i.e. the haze varies with density and depth. In this paper, we propose a self-augmented image dehazing framework, termed D4 (Dehazing via Decomposing transmission map into Density and Depth) for haze generation and removal. Instead of merely estimating transmission maps or clean content, the proposed framework focuses on exploring scattering coefficient and depth information contained in hazy and clean images. With estimated scene depth, our method is capable of re-rendering hazy images with different thick-nesses which further benefits the training of the dehazing network. It is worth noting that the whole training process needs only unpaired hazy and clean images, yet succeeded in recovering the scattering coefficient, depth map and clean content from a single hazy image. Comprehensive experiments demonstrate our method outperforms state-of-the-art unpaired dehazing methods with much fewer parameters and FLOPs. Our code is available at https://github.com/YaN9-Y/D4.
20 citations
References
More filters
Proceedings Article•
01 Jan 2015TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Abstract: We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.
111,197 citations
"Zero-Shot Image Dehazing" refers methods in this paper
...We employ the ADAM optimizer [41] with the default learning rate and the maximal iteration of 500....
[...]
Proceedings Article•
06 Jul 2015TL;DR: Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.
Abstract: Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization, and in some cases eliminates the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.82% top-5 test error, exceeding the accuracy of human raters.
30,843 citations
"Zero-Shot Image Dehazing" refers methods in this paper
...In the decoder, the blocks sequentially perform upsampling, convolution, batch normalization [38], and ReLU activation....
[...]
Proceedings Article•
01 Jan 2014TL;DR: A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.
Abstract: How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case. Our contributions is two-fold. First, we show that a reparameterization of the variational lower bound yields a lower bound estimator that can be straightforwardly optimized using standard stochastic gradient methods. Second, we show that for i.i.d. datasets with continuous latent variables per datapoint, posterior inference can be made especially efficient by fitting an approximate inference model (also called a recognition model) to the intractable posterior using the proposed lower bound estimator. Theoretical advantages are reflected in experimental results.
20,769 citations
"Zero-Shot Image Dehazing" refers background or methods in this paper
...Therefore, to implement our A-Net, we adopt a variational auto-encoder [36] structure which consists of a CNN-based encoder, a symmetric decoder, and an intermedia block....
[...]
...Different from LRec, LA only involves A-Net rather than all the three subnetworks, which aims to disentangle the atmospheric light from x only using variational inference [36]....
[...]
Proceedings Article•
21 Jun 2010TL;DR: Restricted Boltzmann machines were developed using binary stochastic hidden units that learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset.
Abstract: Restricted Boltzmann machines were developed using binary stochastic hidden units. These can be generalized by replacing each binary unit by an infinite number of copies that all have the same weights but have progressively more negative biases. The learning and inference rules for these "Stepped Sigmoid Units" are unchanged. They can be approximated efficiently by noisy, rectified linear units. Compared with binary units, these units learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset. Unlike binary units, rectified linear units preserve information about relative intensities as information travels through multiple layers of feature detectors.
14,799 citations
"Zero-Shot Image Dehazing" refers methods in this paper
...In the encoder, the blocks are composed of a convolutional layer, a ReLU activation function [37], and a max pooling layer in sequence....
[...]
01 Jan 2016
TL;DR: This thesis develops an effective but very simple prior, called the dark channel prior, to remove haze from a single image, and thus solves the ambiguity of the problem.
Abstract: Haze brings troubles to many computer vision/graphics applications. It reduces the visibility of the scenes and lowers the reliability of outdoor surveillance systems; it reduces the clarity of the satellite images; it also changes the colors and decreases the contrast of daily photos, which is an annoying problem to photographers. Therefore, removing haze from images is an important and widely demanded topic in computer vision and computer graphics areas. The main challenge lies in the ambiguity of the problem. Haze attenuates the light reflected from the scenes, and further blends it with some additive light in the atmosphere. The target of haze removal is to recover the reflected light (i.e., the scene colors) from the blended light. This problem is mathematically ambiguous: there are an infinite number of solutions given the blended light. How can we know which solution is true? We need to answer this question in haze removal. Ambiguity is a common challenge for many computer vision problems. In terms of mathematics, ambiguity is because the number of equations is smaller than the number of unknowns. The methods in computer vision to solve the ambiguity can roughly categorized into two strategies. The first one is to acquire more known variables, e.g., some haze removal algorithms capture multiple images of the same scene under different settings (like polarizers).But it is not easy to obtain extra images in practice. The second strategy is to impose extra constraints using some knowledge or assumptions .All the images in this thesis are best viewed in the electronic version. This way is more practical since it requires as few as only one image. To this end, we focus on single image haze removal in this thesis. The key is to find a suitable prior. Priors are important in many computer vision topics. A prior tells the algorithm "what can we know about the fact beforehand" when the fact is not directly available. In general, a prior can be some statistical/physical properties, rules, or heuristic assumptions. The performance of the algorithms is often determined by the extent to which the prior is valid. Some widely used priors in computer vision are the smoothness prior, sparsity prior, and symmetry prior. In this thesis, we develop an effective but very simple prior, called the dark channel prior, to remove haze from a single image. The dark channel prior is a statistical property of outdoor haze-free images: most patches in these images should contain pixels which are dark in at least one color channel. These dark pixels can be due to shadows, colorfulness, geometry, or other factors. This prior provides a constraint for each pixel, and thus solves the ambiguity of the problem. Combining this prior with a physical haze imaging model, we can easily recover high quality haze-free images.
2,055 citations
"Zero-Shot Image Dehazing" refers background or methods in this paper
...(6(a)–6(j))), the input hazy image, DehazeNet [1], MSCNN [3], AOD-Net [6], DCP [2], N2V [31], DIP [2], DCPLoss [16], DDIP [20] and our method are presented....
[...]
...In details, the classic unsupervised approaches are DCP [2], FVR [13], BCCR [12], GRM [40], NLD [11], Noise2Noise (N2N) [30] and DCPLoss [16], and the zero-shot methods are Noise2Void (N2V) [31], DIP [32], DeepDecoder (DD) [33], and Double-DIP (DDIP) [20]....
[...]
...For example, dark channel prior (DCP) [2] assumes that most local patches in outdoor haze-free images have at least one dark channel whose intensity is close to zero....
[...]
...Specifically, ZID proposes a DCP-like loss to train J-Net, whereas YOLY leverages the property of color attenuation prior (CAP)....
[...]
...With such a so-called dark channel loss [2], J-Net incorporates the statistical properties from the recovered “clean images”, thus avoiding an explicit ground truth on the recovered image....
[...]