Showing papers on "Bilateral filter published in 2007"

Kernel Regression for Image Processing and Reconstruction

Abstract: In this paper, we make contact with the field of nonparametric statistics and present a development and generalization of tools and results for use in image processing and reconstruction. In particular, we adapt and expand kernel regression ideas for use in image denoising, upscaling, interpolation, fusion, and more. Furthermore, we establish key relationships with some popular existing methods and show how several of these algorithms, including the recently popularized bilateral filter, are special cases of the proposed framework. The resulting algorithms and analyses are amply illustrated with practical examples

Joint bilateral upsampling

Abstract: Image analysis and enhancement tasks such as tone mapping, colorization, stereo depth, and photomontage, often require computing a solution (e.g., for exposure, chromaticity, disparity, labels) over the pixel grid. Computational and memory costs often require that a smaller solution be run over a downsampled image. Although general purpose upsampling methods can be used to interpolate the low resolution solution to the full resolution, these methods generally assume a smoothness prior for the interpolation. We demonstrate that in cases, such as those above, the available high resolution input image may be leveraged as a prior in the context of a joint bilateral upsampling procedure to produce a better high resolution solution. We show results for each of the applications above and compare them to traditional upsampling methods.

Real-time edge-aware image processing with the bilateral grid

Abstract: We present a new data structure---the bilateral grid, that enables fast edge-aware image processing. By working in the bilateral grid, algorithms such as bilateral filtering, edge-aware painting, and local histogram equalization become simple manipulations that are both local and independent. We parallelize our algorithms on modern GPUs to achieve real-time frame rates on high-definition video. We demonstrate our method on a variety of applications such as image editing, transfer of photographic look, and contrast enhancement of medical images.

Nonlocal Linear Image Regularization and Supervised Segmentation

Abstract: A nonlocal quadratic functional of weighted differences is examined. The weights are based on image features and represent the affinity between different pixels in the image. By prescribing different formulas for the weights, one can generalize many local and nonlocal linear denoising algorithms, including the nonlocal means filter and the bilateral filter. In this framework one can easily show that continuous iterations of the generalized filter obey certain global characteristics and converge to a constant solution. The linear operator associated with the Euler–Lagrange equation of the functional is closely related to the graph Laplacian. We can thus interpret the steepest descent for minimizing the functional as a nonlocal diffusion process. This formulation allows a convenient framework for nonlocal variational minimizations, including variational denoising, Bregman iterations, and the recently proposed inverse scale space. It is also demonstrated how the steepest descent flow can be used for segmenta...

Multiscale shape and detail enhancement from multi-light image collections

Abstract: We present a new image-based technique for enhancing the shape and surface details of an object. The input to our system is a small set of photographs taken from a fixed viewpoint, but under varying lighting conditions. For each image we compute a multiscale decomposition based on the bilateral filter and then reconstruct an enhanced image that combines detail information at each scale across all the input images. Our approach does not require any information about light source positions, or camera calibration, and can produce good results with 3 to 5 input images. In addition our system provides a few high-level parameters for controlling the amount of enhancement and does not require pixel-level user input. We show that the bilateral filter is a good choice for our multiscale algorithm because it avoids the halo artifacts commonly associated with the traditional Laplacian image pyramid. We also develop a new scheme for computing our multiscale bilateral decomposition that is simple to implement, fast O(N2 log N) and accurate.

A Fast Image Super-Resolution Algorithm Using an Adaptive Wiener Filter

Abstract: A computationally simple super-resolution algorithm using a type of adaptive Wiener filter is proposed. The algorithm produces an improved resolution image from a sequence of low-resolution (LR) video frames with overlapping field of view. The algorithm uses subpixel registration to position each LR pixel value on a common spatial grid that is referenced to the average position of the input frames. The positions of the LR pixels are not quantized to a finite grid as with some previous techniques. The output high-resolution (HR) pixels are obtained using a weighted sum of LR pixels in a local moving window. Using a statistical model, the weights for each HR pixel are designed to minimize the mean squared error and they depend on the relative positions of the surrounding LR pixels. Thus, these weights adapt spatially and temporally to changing distributions of LR pixels due to varying motion. Both a global and spatially varying statistical model are considered here. Since the weights adapt with distribution of LR pixels, it is quite robust and will not become unstable when an unfavorable distribution of LR pixels is observed. For translational motion, the algorithm has a low computational complexity and may be readily suitable for real-time and/or near real-time processing applications. With other motion models, the computational complexity goes up significantly. However, regardless of the motion model, the algorithm lends itself to parallel implementation. The efficacy of the proposed algorithm is demonstrated here in a number of experimental results using simulated and real video sequences. A computational analysis is also presented.

Real-time edge-aware image processing with the bilateral grid

Abstract: We present a new data structure---the bilateral grid, that enables fast edge-aware image processing. By working in the bilateral grid, algorithms such as bilateral filtering, edge-aware painting, a...

Bayesian non-local means filter, image redundancy and adaptive dictionaries for noise removal

Abstract: Partial Differential equations (PDE), wavelets-based methods and neighborhood filters were proposed as locally adaptive machines for noise removal Recently, Buades, Coll and Morel proposed the Non-Local (NL-) means filter for image denoising This method replaces a noisy pixel by the weighted average of other image pixels with weights reflecting the similarity between local neighborhoods of the pixel being processed and the other pixels The NL-means filter was proposed as an intuitive neighborhood filter but theoretical connections to diffusion and non-parametric estimation approaches are also given by the authors In this paper we propose another bridge, and show that the NL-means filter also emerges from the Bayesian approach with new arguments Based on this observation, we show how the performance of this filter can be significantly improved by introducing adaptive local dictionaries and a new statistical distance measure to compare patches The new Bayesian NL-means filter is better parametrized and the amount of smoothing is directly determined by the noise variance (estimated from image data) given the patch size Experimental results are given for real images with artificial Gaussian noise added, and for images with real image-dependent noise

Multispectral Bilateral Video Fusion

Abstract: We present a technique for enhancing underexposed visible-spectrum video by fusing it with simultaneously captured video from sensors in nonvisible spectra, such as Short Wave IR or Near IR. Although IR sensors can accurately capture video in low-light and night-vision applications, they lack the color and relative luminances of visible-spectrum sensors. RGB sensors do capture color and correct relative luminances, but are underexposed, noisy, and lack fine features due to short video exposure times. Our enhanced fusion output is a reconstruction of the RGB input assisted by the IR data, not an incorporation of elements imaged only in IR. With a temporal noise reduction, we first remove shot noise and increase the color accuracy of the RGB footage. The IR video is then normalized to ensure cross-spectral compatibility with the visible-spectrum video using ratio images. To aid fusion, we decompose the video sources with edge-preserving filters. We introduce a multispectral version of the bilateral filter called the "dual bilateral" that robustly decomposes the RGB video. It utilizes the less-noisy IR for edge detection but also preserves strong visible-spectrum edges not in the IR. We fuse the RGB low frequencies, the IR texture details, and the dual bilateral edges into a noise-reduced video with sharp details, correct chrominances, and natural relative luminances

Multiscale shape and detail enhancement from multi-light image collections

Abstract: We present a new image-based technique for enhancing the shape and surface details of an object. The input to our system is a small set of photographs taken from a fixed viewpoint, but under varyin...

Bilateral Back-Projection for Single Image Super Resolution

Abstract: In this paper, a novel algorithm for single image super resolution is proposed. Back-projection [1] can minimize the reconstruction error with an efficient iterative procedure. Although it can produce visually appealing result, this method suffers from the chessboard effect and ringing effect, especially along strong edges. The underlining reason is that there is no edge guidance in the error correction process. Bilateral filtering can achieve edge-preserving image smoothing by adding the extra information from the feature domain. The basic idea is to do the smoothing on the pixels which are nearby both in space domain and in feature domain. The proposed bilateral back-projection algorithm strives to integrate the bilateral filtering into the back-projection method. In our approach, the back-projection process can be guided by the edge information to avoid across-edge smoothing, thus the chessboard effect and ringing effect along image edges are removed. Promising results can be obtained by the proposed bilateral back-projection method efficiently.

Adaptive Bilateral Filter for Sharpness Enhancement and Noise Removal

Abstract: In this paper, we present an adaptive bilateral filter (ABF) for sharpness enhancement and noise removal. ABF sharpens an image by increasing the slope of the edges without producing overshoot or undershoot. Our new approach to slope restoration significantly differs from the previous slope restoration algorithms in that ABF does not involve detecting edge orientations or edge profiles. Compared with the bilateral filter, ABF restored images are significantly sharper. Compared with an unsharp mask (USM) based sharpening method the optimal USM (OUM), ABF restored edges are as sharp as those rendered by the OUM, but without halo. ABF also outperforms the bilateral filter and the OUM in noise removal.

Color Pattern And Pixel Level Binning For APS Image Sensor Using 2x2 Photodiode Sharing Scheme

Abstract: A color image sensor includes an array of pixels arranged in a plurality of pixel groups, each pixel group including a floating diffusion that is shared by four pixels disposed in a 2×2 arrangement. Each of said four pixels includes a photodetector and a color filter superposed over the photodetector, wherein a first pair of said four pixels include green color, and a second pair of said four pixels includes either red or blue color filters. A control circuit controls the pixel groups such that discrete image information is generated from each pixel in normal light situations, and such that summed image information is generated from each pixel group in low light situations by simultaneously connecting the green pixels to the floating diffusion during a first time period, and simultaneously connecting the red/blue pixels to said floating diffusion during a second time period.

Unsupervized Video Segmentation With Low Depth of Field

Abstract: In this paper, a novel segmentation algorithm based on matting model is proposed to extract the focused objects in low depth-of-field (DoF) video images. The proposed algorithm is fully automatic and can be used to partition the video image into focused objects and defocused background. This method consists of three stages. The first stage is to generate a saliency map of the input image by the reblurring model. In the second stage, bilateral and morphological filtering are employed to smooth and accentuate the salient regions. Then a trimap with three regions is calculated by an adaptive thresholding method. The third stage involves the proposed adaptive error control matting scheme to extract the boundaries of the focused objects accurately. Experimental evaluation on test sequences shows that the proposed method is capable of segmenting the focused region effectively and accurately.

Techniques for reducing color artifacts in digital images

Abstract: A technique for reducing artifacts in a digital image, in accordance with one embodiment, includes receiving a stream of raw filter pixel data representing the image. The raw filter pixel data is interpolating to produce red, green-on-red row, green-on-blue row and blue pixel data for each pixel. An artifact in one or more given pixels is reduced as a function of a difference between the green-on-red row and green-on-blue row pixel data of each of the given pixels to generate adjusted interpolated pixel data.

Discrete regularization on weighted graphs for image and mesh filtering

Abstract: We propose a discrete regularization framework on weighted graphs of arbitrary topology, which unifies image and mesh filtering. The approach considers the problem as a variational one, which consists in minimizing a weighted sum of two energy terms: a regularization one that uses the discrete p-Laplace operator, and an approximation one. This formulation leads to a family of simple nonlinear filters, parameterized by the degree p of smoothness and by the graph weight function. Some of these filters provide a graph-based version of well-known filters used in image and mesh processing, such as the bilateral filter, the TV digital filter or the nonlocal mean filter.

Improved depth propagation for 2D to 3D video conversion using key-frames

Abstract: The success of introducing 3D television will depend on the availability of high-quality 3D video A possible solution to this problem is to convert existing 2D content into 3D We are working on methods to do this with minimal user interaction and therefore minimal cost In this paper we improve our previous depth propagation that is based on bilateral filtering The proposed improved method uses a two stage approach, where first a per-pixel depth estimate is done using a bilateral filter after which in a second step this estimate is corrected through a block-based motion compensation from the previous frame The best fitting block of the previous frame thus overwrites the depth estimate based on the bilateral filtering The proposed method raises the average Peak- Signal-To-Noise ratio of newly rendered off-centre view images The improvement over 99 propagated frames ranges from 02 up to 43 dB depending on the image sequence (7 pages)

Bilateral Filtering of Diffusion Tensor Magnetic Resonance Images

Abstract: We extend the well-known scalar image bilateral filtering technique to diffusion tensor magnetic resonance images (DTMRI). The scalar version of bilateral image filtering is extended to perform edge-preserving smoothing of DT field data. The bilateral DT filtering is performed in the log-Euclidean framework which guarantees valid output tensors. Smoothing is achieved by weighted averaging of neighboring tensors. Analogous to bilateral filtering of scalar images, the weights are chosen to be inversely proportional to two distance measures: The geometrical Euclidean distance between the spatial locations of tensors and the dissimilarity of tensors. We describe the noniterative DT smoothing equation in closed form and show how interpolation of DT data is treated as a special case of bilateral filtering where only spatial distance is used. We evaluate different DT tensor dissimilarity metrics including the log-Euclidean, the similarity-invariant log-Euclidean, the square root of the J-divergence, and the distance scaled mutual diffusion coefficient. We present qualitative and quantitative smoothing and interpolation results and show their effect on segmentation, for both synthetic DT field data, as well as real cardiac and brain DTMRI data.

Adaptive filter representation

Abstract: The present invention teaches a filter process used in intra or inter prediction of pixel blocks (12). A first, optionally interpolation, filter is applied in the first filter process to get filtered, optionally interpolated, pixel values (11; 21, 23, 25, 27). A differential filter and an adaptive gain are utilized in the second process for improving the prediction performance. The adaptivity of the gain can be made even on block basis, allowing a fine tuning of the pixel prediction and/or a fine tuning of pixel rotation and zooming. Alternatively, a combined one-step filter process using the interpolation filter, the differential filter and the adaptive gain is applied to the pixel values.

Filtering artifacts in images with 3D spatio-temporal fuzzy filters

Abstract: A method filters pixels in a sequence of images. Each image in the sequence is partitioned into blocks of pixels, and the images are processed sequentially. The energy is determined for each block of pixels in each image. The energy of each block is based on variances of intensities of the pixels in the sequence of images. A 3D fuzzy filter is applied to each current pixel in each current block during the sequential processing. The 3D fuzzy filter considers the energy of the block, and the intensities of pixels spatially adjacent and temporally adjacent to the current pixel to remove blocking and ringing artifacts.

System and method for video noise reduction using a unified three-dimensional non-linear filtering

Abstract: The system described herein is a unified approach to three-dimensional filtering for the reduction of video noise. The technique is based on weight averaging pixels of the filter's output value in a three-dimensional neighborhood, in space and time, of the filter's output value. The weights can be computed for individual factors, such as distance, regional differences, etc., or can be combined into a weight that is indicative of all individual weights.

Adaptive image filter for filtering image information

Abstract: This disclosure describes adaptive filtering techniques to improve the quality of captured imagery, such as video or still images. In particular, this disclosure describes adaptive filtering techniques that filter each pixel as a function of a set of surrounding pixels. An adaptive image filter may compare image information associated with a pixel of interest to image information associated with a set of surrounding pixels by, for example, computing differences between the image information associated with the pixel of interest and each of the surrounding pixels of the set. The computed differences can be used in a variety of ways to filter image information of the pixel of interest. In some embodiments, for example, the adaptive image filter may include both a low pass component and high pass component that adjust as a function of the computed differences.

Higher order bilateral filters and their properties

Abstract: Bilateral filtering 1, 2 has proven to be a powerful tool for adaptive denoising purposes. Unlike conventional filters, the bilateral filter defines the closeness of two pixels not only based on geometric distance but also based on radiometric (graylevel) distance. In this paper, to further improve the performance and find new applications, we make contact with a classic non-parametric image reconstruction technique called kernel regression, 3 which is based on local Taylor expansions of the regression function. We extend and generalize the kernel regression method and show that bilateral filtering is a special case of this new class of adaptive image reconstruction techniques, considering a specific choice for weighting kernels and zeroth order Taylor approximation. We show improvements over the classic bilateral filtering can be achieved by using higher order local approximations of the signal.

Method of and apparatus for eliminating image noise

Abstract: A method and apparatus are provided for eliminating image noise to remove spatial-temporal noise and improve visibility. The method includes extracting a spatial-temporal noise level of neighbor pixels around a current pixel, filtering noise of the current pixel by applying a weight to spatial-temporal pixels around the current pixel based on the extracted spatial-temporal noise level, and applying a weight to the noise-filtered pixel and a boosted-up pixel based on an edge intensity and summing the weight-applied pixels. The spatial-temporal noise level is extracted based on spatial-temporal information of neighbor pixels around a current pixel in a current frame and spatial-temporal information of neighbor pixels around a current pixel in a previous frame.

On the Convergence of Bilateral Filter for Edge-Preserving Image Smoothing

Abstract: The bilateral filter represents a wide group of nonlinear filters for edge-preserving image smoothing. In this work, we study the convergence properties of the bilateral filter algorithm. The understanding is established that the bilateral filter is an optimization procedure. We demonstrate that the bilateral filter is equivalent to minimizing a robust cost criterion using iterative reweighting, which is a good approximation to the very fast but unstable Newton's method. Further, the results of the analysis allow us to derive an improved hybrid smoothing scheme with concerns of computational efficiency and edge preservation.

Super-resolution device and method

Abstract: A super-resolution device and method for setting at least one of a plurality of pixels included in image data as target pixels, the image data including pixels arranged in a screen and pixel values representing brightness, an area including the target pixel and peripheral pixels as a target area, and an area for searching pixel value change patterns in the target pixel area; calculating a difference between a first change pattern and second change pattern; comparing a difference between the first and second change patterns; calculating a pixel value of a super-resolution image having a number of pixels larger than a number of pixels included in the image data on the basis of a decimal-accuracy-vector, an extrapolated vector, and pixel values obtained from the image data.

Trained Bilateral Filters and Applications to Coding Artifacts Reduction

Abstract: Bilateral filtering is a simple and non-linear technique to remove the image noise while preserving edges. However, it is difficult to optimize a bilateral filter to obtain desired effect by supervised training. In this paper, we propose a new type of trained bilateral filter, which possesses the essential characteristics of the original bilateral filter and can be optimized offline by least mean square optimization. In applications of JPEG and H.264/MEPG4 AVC deblocking, we compared the proposed filter with the original bilateral filter and other state-of-the-art methods. Experimental results show that the proposed method has a better performance at artifacts reduction and edge preserving.

Spatial and Color Adaptive Gamut Mapping: A Mathematical Framework and Two New Algorithms.

Abstract: A general framework for adaptive gamut mapping is presented in which a wide range of published spatial gamut mapping algorithms fit. Two new spatial and color adaptive gamut mapping algorithms are then introduced. Based on spatial color bilateral filtering, they take into account the color properties of the neighborhood of each pixel. Their goal is to preserve both the color values of the pixels and their relations between neighbors. Results of psychophysical experiments confirm the good performance of the proposed algorithms.

Noise reduction apparatus, method and program for controlling same, image sensor and digital camera

Abstract: Image data pixels indicative of the pixels in a noise-reduction target area having a size of 5×5 pixels is extracted from a plurality of types of CCD-RAW data having red, green and blue color components. A filter for reducing uncorrelated noise is calculated. Uncorrelated noise is removed by performing a filter operation using the calculated filter while correlativity of the CCD-RAW data is maintained. These processing steps are repeated for one frame of CCD-RAW data. After uncorrelated noise has been removed, spatial pixel processing such as an aperture correction is applied.