In this paper, we present an end-to-end network, called Cycle-Dehaze, for single image dehazing problem, which does not require pairs of hazy and corresponding ground truth images for training. That is, we train the network by feeding clean and hazy images in an unpaired manner. Moreover, the proposed approach does not rely on estimation of the atmospheric scattering model parameters. Our method enhances CycleGAN formulation by combining cycle-consistency and perceptual losses in order to improve the quality of textural information recovery and generate visually better haze-free images. Typically, deep learning models for dehazing take low resolution images as input and produce low resolution outputs. However, in the NTIRE 2018 challenge on single image dehazing, high resolution images were provided. Therefore, we apply bicubic downscaling. After obtaining low-resolution outputs from the network, we utilize the Laplacian pyramid to upscale the output images to the original resolution. We conduct experiments on NYU-Depth, I-HAZE, and O-HAZE datasets. Extensive experiments demonstrate that the proposed approach improves CycleGAN method both quantitatively and qualitatively.

/pdf/cycle-dehaze-enhanced-cyclegan-for-single-image-dehazing-37omclvlbm.pdf

Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing

Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing.

The presence of the haze or fog particles in the atmosphere causes visibility degradation in the captured scene. Most of the initial approaches anticipate the transmission map of the hazy scene, airlight component and make use of an atmospheric scattering model to reduce the effect of haze and to recover the haze-free scene. In spite of the remarkable progress of these approaches, they propagate cascaded error upstretched due to the employed priors. We embrace this observation and designed an end-to-end generative adversarial network (GAN) for single image haze removal. Proposed network bypasses the intermediate stages and directly recovers the haze-free scene. Generator architecture of the proposed network is designed using a novel residual inception (RI) module. Proposed RI module comprises of dense connections within the multi-scale convolution layers which allows it to learn the integrated flavors of the haze-related features. Discriminator of the proposed network is built using the dense residual module. Further, to preserve the edge and the structural details in the recovered haze-free scene, structural similarity index and edge loss along with the L1 loss are incorporated in the GAN loss. Experimental analysis has been carried out on NTIRE2019 dehazing challenge dataset, D-Hazy [1] and indoor SOTS [22] databases. Experiments on the publically available datasets show that the proposed method outperforms the existing methods for image de-hazing.

/pdf/ri-gan-an-end-to-end-network-for-single-image-haze-removal-1822y88n3a.pdf

RI-GAN: An End-To-End Network for Single Image Haze Removal

In this paper, we propose a novel image dehazing method. Typical deep learning models for dehazing are trained on paired synthetic indoor dataset. Therefore, these models may be effective for indoor image dehazing but less so for outdoor images. We propose a heterogeneous Generative Adversarial Networks (GAN) based method composed of a cycle-consistent Generative Adversarial Networks (CycleGAN) for producing haze-clear images and a conditional Generative Adversarial Networks (cGAN) for preserving textural details. We introduce a novel loss function in the training of the fused network to minimize GAN generated artifacts, to recover fine details, and to preserve color components. These networks are fused via a convolutional neural network (CNN) to generate dehazed image. Extensive experiments demonstrate that the proposed method significantly outperforms the state-of-the-art methods on both synthetic and real-world hazy images.

Fusion of Heterogeneous Adversarial Networks for Single Image Dehazing

DD-CycleGAN: Unpaired image dehazing via Double-Discriminator Cycle-Consistent Generative Adversarial Network

Single image haze removal is a challenging and ill-posed problem. The existing haze removal methods in literature, including the recently introduced deep learning methods, model the problem of haze removal as that of estimating intermediate parameters, viz., scene transmission map and atmospheric light. These are used to compute the haze-free image from the hazy input image. Such an approach only focuses on accurate estimation of intermediate parameters, while the aesthetic quality of the haze-free image is unaccounted for in the optimization framework. Thus, errors in the estimation of intermediate parameters often lead to generation of inferior quality haze-free images. In this paper, we present CANDY (Conditional Adversarial Networks based Dehazing of hazY images), a fully end-to-end model which directly generates a clean haze-free image from a hazy input image. CANDY also incorporates the visual quality of haze-free image into the optimization function; thus, generating a superior quality haze-free image. This is one of the first works in literature to propose a fully end-to-end model for single image haze removal. Also, this is the first work to explore the concept of generative adversarial networks for the problem of single image haze removal. CANDY was trained on a synthetically created haze image dataset, while evaluation was performed on challenging synthetic as well as real haze image datasets. The extensive evaluation and comparison results of CANDY reveal that it significantly outperforms existing state-of-the-art haze removal methods in literature, both quantitatively as well as qualitatively.

CANDY: Conditional Adversarial Networks based End-to-End System for Single Image Haze Removal

Glare is a hardware problem that occurs because of the light trapped in the lens elements. It is a common problem faced in photography when trying to capture image of a scene having bright source in it or taken in a very bright environment. Glare can hide useful information in the image, can make foreground objects blurry and deformed. In this paper, we propose a novel method to detect glare, mainly focusing on scenario where users take photo of scene having light source in outdoor environment during night. The method described in the paper takes combination of three different masks of original image to detect the glare. First mask is obtained by image segmentation of original image using our improved Bernsen's local thresholding method. To obtain second mask, we binarize the original image by simple thresholding to get specular hot-spot of light present in the original image and for the third mask, we apply thresholding on each RGB channel of original image. Finally, glare is detected using connected component computation on aforementioned masks. The proposed solution detects the glare affected area with very good accuracy.

Detection of glare in night photography

Image orientation detection requires high-level scene understanding. Humans use object recognition and contextual scene information to correctly orient images. In literature, the problem of image orientation detection is mostly confronted by using low-level vision features, while some approaches incorporate few easily detectable semantic cues to gain minor improvements. The vast amount of semantic content in images makes orientation detection challenging, and therefore there is a large semantic gap between existing methods and human behavior. Also, existing methods in literature report highly discrepant detection rates, which is mainly due to large differences in datasets and limited variety of test images used for evaluation. In this work, for the first time, we leverage the power of deep learning and adapt pre-trained convolutional neural networks using largest training dataset to-date for the image orientation detection task. An extensive evaluation of our model on different public datasets shows that it remarkably generalizes to correctly orient a large set of unconstrained images; it also significantly outperforms the state-of-the-art and achieves accuracy very close to that of humans.

Why my photos look sideways or upside down? Detecting canonical orientation of images using convolutional neural networks

Single image haze removal is a very challenging and ill-posed problem. The existing haze removal methods in literature, including the recently introduced deep learning methods, model the problem of haze removal as that of estimating intermediate parameters, viz., scene transmission map and atmospheric light. These are used to compute the haze-free image from the hazy input image. Such an approach only focuses on accurate estimation of intermediate parameters, while the aesthetic quality of the haze-free image is unaccounted for in the optimization framework. Thus, errors in the estimation of intermediate parameters often lead to generation of inferior quality haze-free images. In this paper, we present CANDY (Conditional Adversarial Networks based Dehazing of hazY images), a fully end-to-end model which directly generates a clean haze-free image from a hazy input image. CANDY also incorporates the visual quality of haze-free image into the optimization function; thus, generating a superior quality haze-free image. To the best of our knowledge, this is the first work in literature to propose a fully end-to-end model for single image haze removal. Also, this is the first work to explore the newly introduced concept of generative adversarial networks for the problem of single image haze removal. The proposed model CANDY was trained on a synthetically created haze image dataset, while evaluation was performed on challenging synthetic as well as real haze image datasets. The extensive evaluation and comparison results of CANDY reveal that it significantly outperforms existing state-of-the-art haze removal methods in literature, both quantitatively as well as qualitatively.

CANDY: Conditional Adversarial Networks based Fully End-to-End System for Single Image Haze Removal

Single image depth estimation is a challenging problem. The current state-of-the-art method formulates the problem as that of ordinal regression. However, the formulation is not fully differentiable and depth maps are not generated in an end-to-end fashion. The method uses a native threshold strategy to determine per-pixel depth labels, which results in significant discretization errors. For the first time, we formulate a fully differentiable ordinal regression and train the network in end-to-end fashion. This enables us to include boundary and smoothness constraints in the optimization function, leading to smooth and edge-consistent depth maps. A novel per-pixel confidence map computation for depth refinement is also proposed. Extensive evaluation of the proposed model on challenging benchmarks reveals its superiority over recent state-of-the-art methods, both quantitatively and qualitatively. Additionally, we demonstrate practical utility of the proposed method for single camera bokeh solution using in-house dataset of challenging real-life images.

Kunal Swami

Papers

CANDY: Conditional Adversarial Networks based End-to-End System for Single Image Haze Removal

Detection of glare in night photography

Why my photos look sideways or upside down? Detecting canonical orientation of images using convolutional neural networks

CANDY: Conditional Adversarial Networks based Fully End-to-End System for Single Image Haze Removal

ACED: Accurate And Edge-Consistent Monocular Depth Estimation