scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Image Deblurring via Extreme Channels Prior

Yanyang Yan1, Wenqi Ren1, Yuanfang Guo1, Rui Wang1, Xiaochun Cao1 
01 Jul 2017-pp 6978-6986
TL;DR: This work observes that the bright pixels in the clear images are not likely to be bright after the blur process, and proposes a technique fordeblurring such images which elevates the performance of existing motion deblurring algorithms and takes advantage of both Bright and Dark Channel Prior.
Abstract: Camera motion introduces motion blur, affecting many computer vision tasks. Dark Channel Prior (DCP) helps the blind deblurring on scenes including natural, face, text, and low-illumination images. However, it has limitations and is less likely to support the kernel estimation while bright pixels dominate the input image. We observe that the bright pixels in the clear images are not likely to be bright after the blur process. Based on this observation, we first illustrate this phenomenon mathematically and define it as the Bright Channel Prior (BCP). Then, we propose a technique for deblurring such images which elevates the performance of existing motion deblurring algorithms. The proposed method takes advantage of both Bright and Dark Channel Prior. This joint prior is named as extreme channels prior and is crucial for achieving efficient restorations by leveraging both the bright and dark information. Extensive experimental results demonstrate that the proposed method is more robust and performs favorably against the state-of-the-art image deblurring methods on both synthesized and natural images.

Content maybe subject to copyright    Report

Citations
More filters
Proceedings ArticleDOI
01 Oct 2017
TL;DR: This work presents an algorithm to directly restore a clear highresolution image from a blurry low-resolution input and introduces novel training losses that help recover fine details.
Abstract: We present an algorithm to directly restore a clear highresolution image from a blurry low-resolution input. This problem is highly ill-posed and the basic assumptions for existing super-resolution methods (requiring clear input) and deblurring methods (requiring high-resolution input) no longer hold. We focus on face and text images and adopt a generative adversarial network (GAN) to learn a category-specific prior to solve this problem. However, the basic GAN formulation does not generate realistic highresolution images. In this work, we introduce novel training losses that help recover fine details. We also present a multi-class GAN that can process multi-class image restoration tasks, i.e., face and text images, using a single generator network. Extensive experiments demonstrate that our method performs favorably against the state-of-the-art methods on both synthetic and real-world images at a lower computational cost.

213 citations


Cites methods from "Image Deblurring via Extreme Channe..."

  • ...Several recent methods introduce new image priors that favor clear images over blurred ones in the MAP framework [20, 45, 30, 33, 46]....

    [...]

Proceedings ArticleDOI
18 Jun 2018
TL;DR: Zhang et al. as mentioned in this paper proposed to incorporate global semantic priors as input and impose local structure losses to regularize the output within a multi-scale deep CNN to restore sharp images with more facial details.
Abstract: In this paper, we present an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks (CNNs). As face images are highly structured and share several key semantic components (e.g., eyes and mouths), the semantic information of a face provides a strong prior for restoration. As such, we propose to incorporate global semantic priors as input and impose local structure losses to regularize the output within a multi-scale deep CNN. We train the network with perceptual and adversarial losses to generate photo-realistic results and develop an incremental training strategy to handle random blur kernels in the wild. Quantitative and qualitative evaluations demonstrate that the proposed face deblurring algorithm restores sharp images with more facial details and performs favorably against state-of-the-art methods in terms of restoration quality, face recognition and execution speed.

194 citations

Proceedings ArticleDOI
14 Jun 2020
TL;DR: Experimental results show that the proposed SelfDeblur can achieve notable quantitative gains as well as more visually plausible deblurring results in comparison to state-of-the-art blind deconvolution methods on benchmark datasets and real-world blurry images.
Abstract: Blind deconvolution is a classical yet challenging low-level vision problem with many real-world applications. Traditional maximum a posterior (MAP) based methods rely heavily on fixed and handcrafted priors that certainly are insufficient in characterizing clean images and blur kernels, and usually adopt specially designed alternating minimization to avoid trivial solution. In contrast, existing deep motion deblurring networks learn from massive training images the mapping to clean image or blur kernel, but are limited in handling various complex and large size blur kernels. To connect MAP and deep models, we in this paper present two generative networks for respectively modeling the deep priors of clean image and blur kernel, and propose an unconstrained neural optimization solution to blind deconvolution. In particular, we adopt an asymmetric Autoencoder with skip connections for generating latent clean image, and a fully-connected network (FCN) for generating blur kernel. Moreover, the SoftMax nonlinearity is applied to the output layer of FCN to meet the non-negative and equality constraints. The process of neural optimization can be explained as a kind of ''zero-shot" self-supervised learning of the generative networks, and thus our proposed method is dubbed SelfDeblur. Experimental results show that our SelfDeblur can achieve notable quantitative gains as well as more visually plausible deblurring results in comparison to state-of-the-art blind deconvolution methods on benchmark datasets and real-world blurry images. The source code is publicly available at https://github.com/csdwren/SelfDeblur

184 citations


Cites background or methods from "Image Deblurring via Extreme Channe..."

  • ..., l1/l2-norm [15], patchbased prior [24, 41], low-rank prior [34] and dark channel prior [29, 50] have also been proposed to identify and preserve salient edges for benefiting blur kernel estimation....

    [...]

  • ...Under the MAP-based framework, many fixed and handcrafted regularization terms have been presented for latent clean image and blur kernel [22, 24, 29, 29, 34, 41, 50, 56]....

    [...]

  • ...Although many priors have been suggested for x [2, 15, 27, 56] and k [19, 22, 24, 29, 29, 34, 41, 50, 56], they generally are handcrafted and certainly are insufficient in characterizing clean images and blur kernels....

    [...]

Proceedings ArticleDOI
15 Jun 2019
TL;DR: A blind deblurring method based on Local Maximum Gradient (LMG) prior, inspired by the simple and intuitive observation that the maximum value of a local patch gradient will diminish after the blur process, which is proved to be true both mathematically and empirically.
Abstract: Blind image deblurring aims to recover sharp image from a blurred one while the blur kernel is unknown. To solve this ill-posed problem, a great amount of image priors have been explored and employed in this area. In this paper, we present a blind deblurring method based on Local Maximum Gradient (LMG) prior. Our work is inspired by the simple and intuitive observation that the maximum value of a local patch gradient will diminish after the blur process, which is proved to be true both mathematically and empirically. This inherent property of blur process helps us to establish a new energy function. By introducing an liner operator to compute the Local Maximum Gradient, together with an effective optimization scheme, our method can handle various specific scenarios. Extensive experimental results illustrate that our method is able to achieve favorable performance against state-of-the-art algorithms on both synthetic and real-world images.

132 citations


Cites background or methods or result from "Image Deblurring via Extreme Channe..."

  • ...Our method generates a more visually pleasing result against [27, 28], and contain less ringing artifacts than dark channel based method[20]....

    [...]

  • ...Kernel estimated by extreme channel prior[28] result in large residual blur, and the result generated by our model compares favorably with the method tailored to text [19]....

    [...]

  • ...Recent approaches enforce sparsity on the dark channel [20] and the bright channel [28] of latent images....

    [...]

  • ...(15) with FFT (Fast Fourier Transform) directly, and the solution can be obtained according to [27, 28]....

    [...]

  • ...A number of image priors have also been utilized to solve this ill-posed problem [16, 11, 21, 20, 28, 15]....

    [...]

Journal ArticleDOI
TL;DR: This paper develops a deep learning solution for video dehazing, where a CNN is trained end-to-end to learn how to accumulate information across frames for transmission estimation, and proposes to incorporate global semantic priors as input to regularize the transmission maps.
Abstract: Recent research have shown the potential of using convolutional neural networks (CNNs) to accomplish single image dehazing. In this paper, we take one step further to explore the possibility of exploiting a network to perform haze removal for videos. Unlike single image dehazing, video-based approaches can take advantage of the abundant information that exists across neighboring frames. In this paper, assuming that a scene point yields highly correlated transmission values between adjacent video frames, we develop a deep learning solution for video dehazing, where a CNN is trained end-to-end to learn how to accumulate information across frames for transmission estimation. The estimated transmission map is subsequently used to recover a haze-free frame via atmospheric scattering model. In addition, as the semantic information of a scene provides a strong prior for image restoration, we propose to incorporate global semantic priors as input to regularize the transmission maps so that the estimated maps can be smooth in the regions of the same object and only discontinuous across the boundaries of different objects. To train this network, we generate a dataset consisted of synthetic hazy and haze-free videos for supervision based on the NYU depth dataset. We show that the features learned from this dataset are capable of removing haze that arises in outdoor scenes in a wide range of videos. Extensive experiments demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods on both synthetic and real-world videos.

128 citations


Cites background from "Image Deblurring via Extreme Channe..."

  • ...jointly estimate a transmission map and the underlying hazefree image via designing clear image priors [40], [41]....

    [...]

References
More filters
Journal ArticleDOI
01 Jul 2006
TL;DR: This work introduces a method to remove the effects of camera shake from seriously blurred images, which assumes a uniform camera blur over the image and negligible in-plane camera rotation.
Abstract: Camera shake during exposure leads to objectionable image blur and ruins many photographs. Conventional blind deconvolution methods typically assume frequency-domain constraints on images, or overly simplified parametric forms for the motion path during camera shake. Real camera motions can follow convoluted paths, and a spatial domain prior can better maintain visually salient image characteristics. We introduce a method to remove the effects of camera shake from seriously blurred images. The method assumes a uniform camera blur over the image and negligible in-plane camera rotation. In order to estimate the blur from the camera shake, the user must specify an image region without saturation effects. We show results for a variety of digital photographs taken from personal photo collections.

1,919 citations


"Image Deblurring via Extreme Channe..." refers background or methods in this paper

  • ...Figure 5(a) indicates that our ECP based algorithm performs well against the state-of-the-art methods [2, 5, 19, 12, 16, 25] on this benchmark dataset [13] in terms of cumulative error ratio....

    [...]

  • ...We evaluate the performance of the proposed approach against the state-of-the-art methods [13, 2, 5, 19, 12, 16, 25]....

    [...]

  • ...Such as heavy-tailed gradient distributions [5, 13], normalized sparsity prior [12], L0-regularized gradient [25], patch recurrence prior [15], and a combination of the intensity and gradient prior [16]....

    [...]

Journal ArticleDOI
01 Aug 2008
TL;DR: A new algorithm for removing motion blur from a single image is presented using a unified probabilistic model of both blur kernel estimation and unblurred image restoration and is able to produce high quality deblurred results in low computation time.
Abstract: We present a new algorithm for removing motion blur from a single image. Our method computes a deblurred image using a unified probabilistic model of both blur kernel estimation and unblurred image restoration. We present an analysis of the causes of common artifacts found in current deblurring methods, and then introduce several novel terms within this probabilistic model that are inspired by our analysis. These terms include a model of the spatial randomness of noise in the blurred image, as well a new local smoothness prior that reduces ringing artifacts by constraining contrast in the unblurred image wherever the blurred image exhibits low contrast. Finally, we describe an effficient optimization scheme that alternates between blur kernel estimation and unblurred image restoration until convergence. As a result of these steps, we are able to produce high quality deblurred results in low computation time. We are even able to produce results of comparable quality to techniques that require additional input images beyond a single blurry photograph, and to methods that require additional hardware.

1,338 citations

Proceedings ArticleDOI
23 Jun 2014
TL;DR: This work first develops a fast normalized cuts algorithm, then proposes a high-performance hierarchical segmenter that makes effective use of multiscale information, and proposes a grouping strategy that combines the authors' multiscales regions into highly-accurate object candidates by exploring efficiently their combinatorial space.
Abstract: We propose a unified approach for bottom-up hierarchical image segmentation and object candidate generation for recognition, called Multiscale Combinatorial Grouping (MCG). For this purpose, we first develop a fast normalized cuts algorithm. We then propose a high-performance hierarchical segmenter that makes effective use of multiscale information. Finally, we propose a grouping strategy that combines our multiscale regions into highly-accurate object candidates by exploring efficiently their combinatorial space. We conduct extensive experiments on both the BSDS500 and on the PASCAL 2012 segmentation datasets, showing that MCG produces state-of-the-art contours, hierarchical regions and object candidates.

1,279 citations


"Image Deblurring via Extreme Channe..." refers methods in this paper

  • ...We further validate our observation with the statistics of the bright channels by randomly selecting 5000 images from the PASCAL 2012 dataset [1] without any manually pre-processing....

    [...]

Journal ArticleDOI
TL;DR: A blind deconvolution algorithm based on the total variational (TV) minimization method proposed is presented, and it is remarked that psf's without sharp edges, e.g., Gaussian blur, can also be identified through the TV approach.
Abstract: We present a blind deconvolution algorithm based on the total variational (TV) minimization method proposed by Acar and Vogel (1994). The motivation for regularizing with the TV norm is that it is extremely effective for recovering edges of images as well as some blurring functions, e.g., motion blur and out-of-focus blur. An alternating minimization (AM) implicit iterative scheme is devised to recover the image and simultaneously identify the point spread function (PSF). Numerical results indicate that the iterative scheme is quite robust, converges very fast (especially for discontinuous blur), and both the image and the PSF can be recovered under the presence of high noise level. Finally, we remark that PSFs without sharp edges, e.g., Gaussian blur, can also be identified through the TV approach.

1,220 citations


"Image Deblurring via Extreme Channe..." refers methods in this paper

  • ...To estimate blur kernels from blurry images, other existing approaches utilize statistical priors[4, 20, 14, 12, 25, 16] or additional information [8, 3, 9] to solve the ill-posed problem....

    [...]

Proceedings ArticleDOI
20 Jun 2009
TL;DR: The previously reported failure of the naive MAP approach is explained by demonstrating that it mostly favors no-blur explanations and it is shown that since the kernel size is often smaller than the image size a MAP estimation of the kernel alone can be well constrained and accurately recover the true blur.
Abstract: Blind deconvolution is the recovery of a sharp version of a blurred image when the blur kernel is unknown. Recent algorithms have afforded dramatic progress, yet many aspects of the problem remain challenging and hard to understand. The goal of this paper is to analyze and evaluate recent blind deconvolution algorithms both theoretically and experimentally. We explain the previously reported failure of the naive MAP approach by demonstrating that it mostly favors no-blur explanations. On the other hand we show that since the kernel size is often smaller than the image size a MAP estimation of the kernel alone can be well constrained and accurately recover the true blur. The plethora of recent deconvolution techniques makes an experimental evaluation on ground-truth data important. We have collected blur data with ground truth and compared recent algorithms under equal settings. Additionally, our data demonstrates that the shift-invariant blur assumption made by most algorithms is often violated.

1,219 citations


"Image Deblurring via Extreme Channe..." refers background or methods in this paper

  • ...In addition, we also show the performances of the proposed algorithm against the DCP-based [17] and BCPbased methods on this benchmark dataset [13] in terms of the cumulative error ratio....

    [...]

  • ...Such as heavy-tailed gradient distributions [5, 13], normalized sparsity prior [12], L0-regularized gradient [25], patch recurrence prior [15], and a combination of the intensity and gradient prior [16]....

    [...]

  • ...To better verify the effectiveness of our proposed method, we use the image benchmark datasets [13, 11] for quantitative evaluations and follow the protocols of [13, 11] for fair comparisons....

    [...]

  • ...Quantitative evaluations on the benchmark datasets by [13] and [11], respectively....

    [...]

  • ...Quantitative evaluations on the benchmark dataset by [13] with and without using DCP or BCP....

    [...]