scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles

TL;DR: The key idea is to formulate the forward amplitude modulated light propagation as a convolution with custom codes, record samples by introducing a simple sequence of electronic time delays, and perform sparse deconvolution to recover sequences of Diracs that correspond to multipath returns.
Abstract: Time of flight cameras produce real-time range maps at a relatively low cost using continuous wave amplitude modulation and demodulation. However, they are geared to measure range (or phase) for a single reflected bounce of light and suffer from systematic errors due to multipath interference.We re-purpose the conventional time of flight device for a new goal: to recover per-pixel sparse time profiles expressed as a sequence of impulses. With this modification, we show that we can not only address multipath interference but also enable new applications such as recovering depth of near-transparent surfaces, looking through diffusers and creating time-profile movies of sweeping light.Our key idea is to formulate the forward amplitude modulated light propagation as a convolution with custom codes, record samples by introducing a simple sequence of electronic time delays, and perform sparse deconvolution to recover sequences of Diracs that correspond to multipath returns. Applications to computer vision include ranging of near-transparent objects and subsurface imaging through diffusers. Our low cost prototype may lead to new insights regarding forward and inverse problems in light transport.

Content maybe subject to copyright    Report

Citations
More filters
Journal ArticleDOI
TL;DR: A detailed and in-depth comparison between both Kinect Structured-Light with Kinect Time-of-Flight cameras to provide a solid insight into the pros and cons of either device.

323 citations

Journal ArticleDOI
TL;DR: A non-line-of-sight imaging system that uses a single-pixel, single-photon avalanche diode (SPAD) to collect time- of-flight information that provides significant improvements in terms of power requirements, form factor, cost, and reconstruction time, while maintaining a comparable time resolution.
Abstract: By using time-of-flight information encoded in multiply scattered light, it is possible to reconstruct images of objects hidden from the camera’s direct line of sight. Here, we present a non-line-of-sight imaging system that uses a single-pixel, single-photon avalanche diode (SPAD) to collect time-of-flight information. Compared to earlier systems, this modification provides significant improvements in terms of power requirements, form factor, cost, and reconstruction time, while maintaining a comparable time resolution. The potential for further size and cost reduction of this technology make this system a good base for developing a practical system that can be used in real world applications.

236 citations

Journal ArticleDOI
20 Apr 2020
TL;DR: In this paper, a 1 Mpixel single-photon avalanche diode camera with 3.8 ns time gating and 24 kfps frame rate is presented, fabricated in 180 nm CMOS image sensor technology.
Abstract: We present a 1 Mpixel single-photon avalanche diode camera featuring 3.8 ns time gating and 24 kfps frame rate, fabricated in 180 nm CMOS image sensor technology. We designed two pixels with a pitch of 9.4 µm in 7 T and 5.75 T configurations respectively, achieving a maximum fill factor of 13.4%. The maximum photon detection probability is 27%, median dark count rate is 2.0 cps, variation in gating length is 120 ps, position skew is 410 ps, and rise/fall time is ${ \lt }{550}\;{\rm ps}$<550ps, all FWHM at 3.3 V excess bias. The sensor was used to capture 2D/3D scenes over 2 m with resolution (least significant bit) of 5.4 mm and precision better than 7.8 mm (rms). We demonstrate extended dynamic range in dual exposure operation mode and show spatially overlapped multi-object detection in single-photon time-gated time-of-flight experiments.

156 citations

Journal ArticleDOI
TL;DR: The requirements for a simple modern particle tracking microrheology experiment are introduced, the error analysis methods associated with it and the mathematical techniques required to calculate the linear viscoelasticity are discussed.
Abstract: New developments in the microrheology of complex fluids are considered. Firstly the requirements for a simple modern particle tracking microrheology experiment are introduced, the error analysis methods associated with it and the mathematical techniques required to calculate the linear viscoelasticity. Progress in microrheology instrumentation is then described with respect to detectors, light sources, colloidal probes, magnetic tweezers, optical tweezers, diffusing wave spectroscopy, optical coherence tomography, fluorescence correlation spectroscopy, elastic- and quasi-elastic scattering techniques, 3D tracking, single molecule methods, modern microscopy methods and microfluidics. New theoretical techniques are also reviewed such as Bayesian analysis, oversampling, inversion techniques, alternative statistical tools for tracks (angular correlations, first passage probabilities, the kurtosis, motor protein step segmentation etc), issues in micro/macro rheological agreement and two particle methodologies. Applications where microrheology has begun to make some impact are also considered including semi-flexible polymers, gels, microorganism biofilms, intracellular methods, high frequency viscoelasticity, comb polymers, active motile fluids, blood clots, colloids, granular materials, polymers, liquid crystals and foods. Two large emergent areas of microrheology, non-linear microrheology and surface microrheology are also discussed.

154 citations


Additional excerpts

  • ...The responses of time locked cameras (time of flight cameras) has been demonstrated at picosecond and femtosecond time scales using relatively cheap apparatus (<$500/£400) [69]....

    [...]

Journal ArticleDOI
TL;DR: In this article, the authors used terahertz time-gated spectral imaging to extract occluded text from paper pages with subwavelength spacing, which may be used to non-destructively detect and study defects and structures within materials.
Abstract: Terahertz radiation may be used to nondestructively detect and study defects and structures within materials. Here the authors use terahertz time-gated spectral imaging to extract occluded text from paper pages with subwavelength spacing.

144 citations

References
More filters
Journal ArticleDOI
TL;DR: In this article, a procedure for representing a times series as the sum of a smoothly varying trend component and a cyclical component is proposed, and the nature of the comovements of the cyclical components of a variety of macroeconomic time series is documented.
Abstract: A study documents some features of aggregate economic fluctuations sometimes referred to as business cycles. The investigation uses quarterly data from the postwar US economy. The fluctuations studied are those that are too rapid to be accounted for by slowly changing demographic and technological factors and changes in the stocks of capital that produce secular growth in output per capita. The study proposes a procedure for representing a times series as the sum of a smoothly varying trend component and a cyclical component. The nature of the comovements of the cyclical components of a variety of macroeconomic time series is documented. It is found that these comovements are very different than the corresponding comovements of the slowly varying trend components.

5,998 citations

Journal ArticleDOI
TL;DR: It is proved that replacing the usual quadratic regularizing penalties by weighted 𝓁p‐penalized penalties on the coefficients of such expansions, with 1 ≤ p ≤ 2, still regularizes the problem.
Abstract: We consider linear inverse problems where the solution is assumed to have a sparse expansion on an arbitrary preassigned orthonormal basis. We prove that replacing the usual quadratic regularizing penalties by weighted p-penalties on the coefficients of such expansions, with 1 ≤ p ≤ 2, still regularizes the problem. Use of such p-penalized problems with p < 2 is often advocated when one expects the underlying ideal noiseless solution to have a sparse expansion with respect to the basis under consideration. To compute the corresponding regularized solutions, we analyze an iterative algorithm that amounts to a Landweber iteration with thresholding (or nonlinear shrinkage) applied at each iteration step. We prove that this algorithm converges in norm. © 2004 Wiley Periodicals, Inc.

4,339 citations


"Coded time of flight cameras: spars..." refers background in this paper

  • ...Since then a number of modifications have been proposed for both the cost function Jλ, for example see [O’Brien et al. 1994] as well as the problem of accelerating the optimization problem with `1 penalty term [Darche 1989; Daubechies et al. 2004] with Lasso and Basis–pursuit being notable examples....

    [...]

  • ...1994] as well as the problem of accelerating the optimization problem with `1 penalty term [Darche 1989; Daubechies et al. 2004] with Lasso and Basis–pursuit being notable examples....

    [...]

Posted Content
Abstract: We consider linear inverse problems where the solution is assumed to have a sparse expansion on an arbitrary pre-assigned orthonormal basis. We prove that replacing the usual quadratic regularizing penalties by weighted l^p-penalties on the coefficients of such expansions, with 1 < or = p < or =2, still regularizes the problem. If p < 2, regularized solutions of such l^p-penalized problems will have sparser expansions, with respect to the basis under consideration. To compute the corresponding regularized solutions we propose an iterative algorithm that amounts to a Landweber iteration with thresholding (or nonlinear shrinkage) applied at each iteration step. We prove that this algorithm converges in norm. We also review some potential applications of this method.

3,640 citations

Proceedings ArticleDOI
01 Aug 2001
TL;DR: A simple model for subsurface light transport in translucent materials that enables efficient simulation of effects that BRDF models cannot capture and a new, rapid image-based measurement technique for determining the optical properties of translucent materials.
Abstract: This paper introduces a simple model for subsurface light transport in translucent materials. The model enables efficient simulation of effects that BRDF models cannot capture, such as color bleeding within materials and diffusion of light across shadow boundaries. The technique is efficient even for anisotropic, highly scattering media that are expensive to simulate using existing methods. The model combines an exact solution for single scattering with a dipole point source diffusion approximation for multiple scattering. We also have designed a new, rapid image-based measurement technique for determining the optical properties of translucent materials. We validate the model by comparing predicted and measured values and show how the technique can be used to recover the optical properties of a variety of materials, including milk, marble, and skin. Finally, we describe sampling techniques that allow the model to be used within a conventional ray tracer.

919 citations


"Coded time of flight cameras: spars..." refers background in this paper

  • ...It is well known that accurate modelling of global illumination is integral to realistic renderings [Jensen et al. 2001]....

    [...]