In this paper, we propose a novel explicit image filter called guided filter. Derived from a local linear model, the guided filter computes the filtering output by considering the content of a guidance image, which can be the input image itself or another different image. The guided filter can be used as an edge-preserving smoothing operator like the popular bilateral filter [1], but it has better behaviors near edges. The guided filter is also a more generic concept beyond smoothing: It can transfer the structures of the guidance image to the filtering output, enabling new filtering applications like dehazing and guided feathering. Moreover, the guided filter naturally has a fast and nonapproximate linear time algorithm, regardless of the kernel size and the intensity range. Currently, it is one of the fastest edge-preserving filters. Experiments show that the guided filter is both effective and efficient in a great variety of computer vision and computer graphics applications, including edge-aware smoothing, detail enhancement, HDR compression, image matting/feathering, dehazing, joint upsampling, etc.

/pdf/guided-image-filtering-5atmpsksoa.pdf

Guided Image Filtering

Many recent computational photography techniques decompose an image into a piecewise smooth base layer, containing large scale variations in intensity, and a residual detail layer capturing the smaller scale details in the image. In many of these applications, it is important to control the spatial scale of the extracted details, and it is often desirable to manipulate details at multiple scales, while avoiding visual artifacts.In this paper we introduce a new way to construct edge-preserving multi-scale image decompositions. We show that current basedetail decomposition techniques, based on the bilateral filter, are limited in their ability to extract detail at arbitrary scales. Instead, we advocate the use of an alternative edge-preserving smoothing operator, based on the weighted least squares optimization framework, which is particularly well suited for progressive coarsening of images and for multi-scale detail extraction. After describing this operator, we show how to use it to construct edge-preserving multi-scale decompositions, and compare it to the bilateral filter, as well as to other schemes. Finally, we demonstrate the effectiveness of our edge-preserving decompositions in the context of LDR and HDR tone mapping, detail enhancement, and other applications.

/pdf/edge-preserving-decompositions-for-multi-scale-tone-and-4ckfmqmulb.pdf

Edge-preserving decompositions for multi-scale tone and detail manipulation

Image analysis and enhancement tasks such as tone mapping, colorization, stereo depth, and photomontage, often require computing a solution (e.g., for exposure, chromaticity, disparity, labels) over the pixel grid. Computational and memory costs often require that a smaller solution be run over a downsampled image. Although general purpose upsampling methods can be used to interpolate the low resolution solution to the full resolution, these methods generally assume a smoothness prior for the interpolation. We demonstrate that in cases, such as those above, the available high resolution input image may be leveraged as a prior in the context of a joint bilateral upsampling procedure to produce a better high resolution solution. We show results for each of the applications above and compare them to traditional upsampling methods.

/pdf/joint-bilateral-upsampling-2nmjsi8mix.pdf

Joint bilateral upsampling

We present a new image editing method, particularly effective for sharpening major edges by increasing the steepness of transition while eliminating a manageable degree of low-amplitude structures. The seemingly contradictive effect is achieved in an optimization framework making use of L0 gradient minimization, which can globally control how many non-zero gradients are resulted in to approximate prominent structure in a sparsity-control manner. Unlike other edge-preserving smoothing approaches, our method does not depend on local features, but instead globally locates important edges. It, as a fundamental tool, finds many applications and is particularly beneficial to edge extraction, clip-art JPEG artifact removal, and non-photorealistic effect generation.

/pdf/image-smoothing-via-l0-gradient-minimization-e3zx7ks7rc.pdf

Image smoothing via L0 gradient minimization

The bilateral filter is a nonlinear filter that smoothes a signal while preserving strong edges. It has demonstrated great effectiveness for a variety of problems in computer vision and computer graphics, and fast versions have been proposed. Unfortunately, little is known about the accuracy of such accelerations. In this paper, we propose a new signal-processing analysis of the bilateral filter which complements the recent studies that analyzed it as a PDE or as a robust statistical estimator. The key to our analysis is to express the filter in a higher-dimensional space where the signal intensity is added to the original domain dimensions. Importantly, this signal-processing perspective allows us to develop a novel bilateral filtering acceleration using downsampling in space and intensity. This affords a principled expression of accuracy in terms of bandwidth and sampling. The bilateral filter can be expressed as linear convolutions in this augmented space followed by two simple nonlinearities. This allows us to derive criteria for downsampling the key operations and achieving important acceleration of the bilateral filter. We show that, for the same running time, our method is more accurate than previous acceleration techniques. Typically, we are able to process a 2 megapixel image using our acceleration technique in less than a second, and have the result be visually similar to the exact computation that takes several tens of minutes. The acceleration is most effective with large spatial kernels. Furthermore, this approach extends naturally to color images and cross bilateral filtering.

/pdf/a-fast-approximation-of-the-bilateral-filter-using-a-signal-1pcp5jagu7.pdf

A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach

Median filtering is a cornerstone of modern image processing and is used extensively in smoothing and de-noising applications. The fastest commercial implementations (e.g. in Adobe® Photoshop® CS2) exhibit O(r) runtime in the radius of the filter, which limits their usefulness in realtime or resolution-independent contexts. We introduce a CPU-based, vectorizable O(log r) algorithm for median filtering, to our knowledge the most efficient yet developed. Our algorithm extends to images of any bit-depth, and can also be adapted to perform bilateral filtering. On 8-bit data our median filter outperforms Photoshop's implementation by up to a factor of fifty.

Fast median and bilateral filtering

The invention is a method and apparatus for improving image-processing applications. Embodiments of the invention provide methods for preserving computation results and offer intermediary computation steps to allow the processing of images at any location to take advantage of previously processed image areas. The preferred embodiment offers a method for building a median filter that significantly improves the processing speed over basic techniques. By building a histogram hierarchy, image data statistics are added and subtracted using a multiplicity of histograms from the histogram hierarchy, where each histogram describes an image area. Furthermore, a histogram hierarchy is built using multiple layers, each layer defining a level of statistical resolution.

Method and apparatus for processing image data

The present invention is a method and system for deriving histograms for neighborhoods of pixels in an image that uses subtractive image regions to increase the size of overlapping regions of regions of series of adjacent image pixels to a size equal to the size of a full neighborhood. Each neighborhood (except for a central neighborhood) is viewed as a combination of the central overlapping neighborhood and one or more additional regions, at least one of which comprises a “subtractive” region, and which vary in size depending on the distance of a pixel from the pixel that is the basis for the central neighborhood. The histogram for each neighborhood is derived by adding the histograms of the central neighborhood and the histograms for the relevant additional region or regions, which may have one or more negative element values. The method and system of the invention allows the processing time for implementing a median filter to be improved to approximately the order of log(r), where r is the “radius” of the median filter being applied.

Ben Weiss

Papers

Fast median and bilateral filtering

Method and apparatus for processing image data

Method and system for improved processing of image data