Resolving focal plane ambiguity in depth map creation from defocus blur using chromatic aberration
01 Dec 2015-pp 1-5
TL;DR: This paper presents a method to resolve focal plane ambiguity in depth map creation from defocus blur with the help of Chromatic Aberration (CA).
Abstract: Focal plane ambiguity in depth map creation from defocus blur has remained an challenging problem. In this paper, we present a method to resolve this issue with the help of Chromatic Aberration(CA). CA is a distortion referred to focal length variation of the lens with wavelength of light. When light, a mixture of various monochromatic components, passes through a lens, multiple focal planes are generated due to CA. There also exists an inherent ordering in the defocus blur of the object for different RGB components depending on whether the object lies in the near or far focus region. The ordering reverses as soon as focal planes are crossed. By using this difference in ordering of the defocus amount for different RGB components of an object, we can deduce whether the object is present in front of or behind the image plane, hence using this information, a more reliable depth map can be obtained.
Citations
More filters
TL;DR: This paper presents a novel framework to generate a more accurate depth map for video using defocus and motion cues and corrects the errors in other parts of depth map caused by inaccurate estimation of defocus blur and motion.
Abstract: Significant recent developments in 3D display technology have focused on techniques for converting 2D media into 3D. Depth map is an integral part of 2D-to-3D conversion. Combining multiple depth cues results in a more accurate depth map as it compensates for the errors caused by one depth cue as well as its absence by other depth cues. In this paper, we present a novel framework to generate a more accurate depth map for video using defocus and motion cues. The moving objects present in the scene are the source of errors in both defocus and motion-based depth map estimation. The proposed method rectifies these errors in the depth map by integrating defocus blur and motion cues. In addition, it also corrects the errors in other parts of depth map caused by inaccurate estimation of defocus blur and motion. Since the proposed integration approach relies on the characteristics of point spread functions of defocus and motion blur along with their relations to camera parameters, it is more accurate and reliable.
14 citations
Cites background from "Resolving focal plane ambiguity in ..."
...Change in camera focus is a common phenomenon in video, which can be corrected using [37]....
[...]
TL;DR: A novel method to estimate the concurrent defocus and motion blurs in a single image is proposed, which works well for real images as well as for compressed images.
Abstract: The occurrence of motion blur along with defocus blur is a common phenomena in natural images. Usually, these blurs are spatially varying in nature for any general image and estimation of one type of blur is affected by presence of other. In this paper, we propose a novel method to estimate the concurrent defocus and motion blurs in a single image. Unlike the recent methods, which perform well only on simulated conditions or in presence of single type of blur, proposed method works well for real images as well as for compressed images. In this paper, we consider only commonly associated motion and defocus blurs for analysis. Decoupling of motion and defocus blur provides a fundamental tool that can be used for various analysis and applications.
11 citations
01 Jan 2018
TL;DR: It is shown that the metric scale can be estimated using information gathered through monocularSLAM and image blur due to defocus and a nonlinear least squares optimization problem is formulated to integrate depth estimates from defocus to monocular SLAM.
Abstract: This letter presents a novel approach to correct errors caused by accumulated scale drift in monocular SLAM. It is shown that the metric scale can be estimated using information gathered through monocular SLAM and image blur due to defocus. A nonlinear least squares optimization problem is formulated to integrate depth estimates from defocus to monocular SLAM. An algorithm to process the output keyframe and feature location estimates generated by a monocular SLAM algorithm to correct for scale drift at selected local regions of the environment is presented. The proposed algorithm is experimentally evaluated by processing the output of ORB-SLAM to obtain accurate metric scale maps from a monocular camera without any prior knowledge about the scene.
3 citations
TL;DR: The authors propose a blur parameter locus curve (BPLC) as a system representation which has a one to one relationship with blur and characterise the PSF by decomposing the variation of BPLC across all directions based on the study performed for different possible forms of the blur kernels.
Abstract: Conventionally, the point spread function (PSF) is understood as a characteristic function of any optical system. It captures the information about the amount of blur present along all the directions for a point in the scene. However, the dependence of blur on the PSF is in the form of convolution for any object other than a point source present in the scene and hence their relationship is less explicit. The authors propose a blur parameter locus curve (BPLC) as a system representation which has a one to one relationship with blur. BPLC simply is a chart of blur amounts in all directions of a given PSF with respect to the selected measurement function. They further characterise the PSF by decomposing the variation of BPLC across all directions based on the study performed for different possible forms of the blur kernels. Such decomposition provides powerful tools for various analysis. As PSF can be anisotropic, the computation of BPLC becomes an essential intermediate step to obtain the scale map as at the same scale, blur is different in different directions. Furthermore, they demonstrate the use of BPLC to obtain other system characteristics function such as PSF.
1 citations
01 Sep 2017
TL;DR: It is shown in real experiments that the proposed approach has the potential to enhance robot navigation algorithms that rely on monocular cameras and converges to a metric scale, accurate, sparse depth map and 3D camera poses with images from a monocular camera.
Abstract: This paper presents a novel approach to metric scale reconstruction of a three-dimensional (3D) scene using a monocular camera. Using a sequence of images from a monocular camera with a fixed focus lens, metric distance to a set of features in the environment is estimated from image blur due to defocus. The blur texture ambiguity which causes scale errors in depth from defocus is corrected in an EKF framework that exploits image velocity measurements. We show in real experiments that our method converges to a metric scale, accurate, sparse depth map and 3D camera poses with images from a monocular camera. Therefore, the proposed approach has the potential to enhance robot navigation algorithms that rely on monocular cameras.
1 citations
Cites background from "Resolving focal plane ambiguity in ..."
...[16] demonstrated that chromatic aberration provides an effective indicator to solve the focal plane ambiguity....
[...]
References
More filters
TL;DR: A simple but effective image prior - dark channel prior to remove haze from a single input image is proposed, based on a key observation - most local patches in haze-free outdoor images contain some pixels which have very low intensities in at least one color channel.
Abstract: In this paper, we propose a simple but effective image prior-dark channel prior to remove haze from a single input image. The dark channel prior is a kind of statistics of outdoor haze-free images. It is based on a key observation-most local patches in outdoor haze-free images contain some pixels whose intensity is very low in at least one color channel. Using this prior with the haze imaging model, we can directly estimate the thickness of the haze and recover a high-quality haze-free image. Results on a variety of hazy images demonstrate the power of the proposed prior. Moreover, a high-quality depth map can also be obtained as a byproduct of haze removal.
3,668 citations
01 Jan 2016
TL;DR: This thesis develops an effective but very simple prior, called the dark channel prior, to remove haze from a single image, and thus solves the ambiguity of the problem.
Abstract: Haze brings troubles to many computer vision/graphics applications. It reduces the visibility of the scenes and lowers the reliability of outdoor surveillance systems; it reduces the clarity of the satellite images; it also changes the colors and decreases the contrast of daily photos, which is an annoying problem to photographers. Therefore, removing haze from images is an important and widely demanded topic in computer vision and computer graphics areas. The main challenge lies in the ambiguity of the problem. Haze attenuates the light reflected from the scenes, and further blends it with some additive light in the atmosphere. The target of haze removal is to recover the reflected light (i.e., the scene colors) from the blended light. This problem is mathematically ambiguous: there are an infinite number of solutions given the blended light. How can we know which solution is true? We need to answer this question in haze removal. Ambiguity is a common challenge for many computer vision problems. In terms of mathematics, ambiguity is because the number of equations is smaller than the number of unknowns. The methods in computer vision to solve the ambiguity can roughly categorized into two strategies. The first one is to acquire more known variables, e.g., some haze removal algorithms capture multiple images of the same scene under different settings (like polarizers).But it is not easy to obtain extra images in practice. The second strategy is to impose extra constraints using some knowledge or assumptions .All the images in this thesis are best viewed in the electronic version. This way is more practical since it requires as few as only one image. To this end, we focus on single image haze removal in this thesis. The key is to find a suitable prior. Priors are important in many computer vision topics. A prior tells the algorithm "what can we know about the fact beforehand" when the fact is not directly available. In general, a prior can be some statistical/physical properties, rules, or heuristic assumptions. The performance of the algorithms is often determined by the extent to which the prior is valid. Some widely used priors in computer vision are the smoothness prior, sparsity prior, and symmetry prior. In this thesis, we develop an effective but very simple prior, called the dark channel prior, to remove haze from a single image. The dark channel prior is a statistical property of outdoor haze-free images: most patches in these images should contain pixels which are dark in at least one color channel. These dark pixels can be due to shadows, colorfulness, geometry, or other factors. This prior provides a constraint for each pixel, and thus solves the ambiguity of the problem. Combining this prior with a physical haze imaging model, we can easily recover high quality haze-free images.
2,055 citations
"Resolving focal plane ambiguity in ..." refers background in this paper
...Depth from atmospheric haze [2] can be applied only when the image meets the requirement of scene having hazy environment....
[...]
TL;DR: A closed-form solution to natural image matting that allows us to find the globally optimal alpha matte by solving a sparse linear system of equations and predicts the properties of the solution by analyzing the eigenvectors of a sparse matrix, closely related to matrices used in spectral image segmentation algorithms.
Abstract: Interactive digital matting, the process of extracting a foreground object from an image based on limited user input, is an important task in image and video editing. From a computer vision perspective, this task is extremely challenging because it is massively ill-posed - at each pixel we must estimate the foreground and the background colors, as well as the foreground opacity ("alpha matte") from a single color measurement. Current approaches either restrict the estimation to a small part of the image, estimating foreground and background colors based on nearby pixels where they are known, or perform iterative nonlinear estimation by alternating foreground and background color estimation with alpha estimation. In this paper, we present a closed-form solution to natural image matting. We derive a cost function from local smoothness assumptions on foreground and background colors and show that in the resulting expression, it is possible to analytically eliminate the foreground and background colors to obtain a quadratic cost function in alpha. This allows us to find the globally optimal alpha matte by solving a sparse linear system of equations. Furthermore, the closed-form formula allows us to predict the properties of the solution by analyzing the eigenvectors of a sparse matrix, closely related to matrices used in spectral image segmentation algorithms. We show that high-quality mattes for natural images may be obtained from a small amount of user input.
1,851 citations
29 Jul 2007
TL;DR: A simple modification to a conventional camera is proposed to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture, and introduces a criterion for depth discriminability which is used to design the preferred aperture pattern.
Abstract: A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modification to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocusing, or re-rendering of the scene from an alternate viewpoint.
1,489 citations
Proceedings Article•
05 Dec 2005TL;DR: This work begins by collecting a training set of monocular images (of unstructured outdoor environments which include forests, trees, buildings, etc.) and their corresponding ground-truth depthmaps, and applies supervised learning to predict the depthmap as a function of the image.
Abstract: We consider the task of depth estimation from a single monocular image. We take a supervised learning approach to this problem, in which we begin by collecting a training set of monocular images (of unstructured outdoor environments which include forests, trees, buildings, etc.) and their corresponding ground-truth depthmaps. Then, we apply supervised learning to predict the depthmap as a function of the image. Depth estimation is a challenging problem, since local features alone are insufficient to estimate depth at a point, and one needs to consider the global context of the image. Our model uses a discriminatively-trained Markov Random Field (MRF) that incorporates multiscale local- and global-image features, and models both depths at individual points as well as the relation between depths at different points. We show that, even on unstructured scenes, our algorithm is frequently able to recover fairly accurate depthmaps.
1,079 citations
"Resolving focal plane ambiguity in ..." refers background in this paper
...The implementation involving texture needs prior training for creating the depth map [1]....
[...]