Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Kinect range sensing

https://shura.shu.ac.uk/7180/1/Rodriggues_%2D_optical_techniques_%2D_Review_MIA_revision_v9.pdf

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery.

Time of flight cameras produce real-time range maps at a relatively low cost using continuous wave amplitude modulation and demodulation. However, they are geared to measure range (or phase) for a single reflected bounce of light and suffer from systematic errors due to multipath interference.We re-purpose the conventional time of flight device for a new goal: to recover per-pixel sparse time profiles expressed as a sequence of impulses. With this modification, we show that we can not only address multipath interference but also enable new applications such as recovering depth of near-transparent surfaces, looking through diffusers and creating time-profile movies of sweeping light.Our key idea is to formulate the forward amplitude modulated light propagation as a convolution with custom codes, record samples by introducing a simple sequence of electronic time delays, and perform sparse deconvolution to recover sequences of Diracs that correspond to multipath returns. Applications to computer vision include ranging of near-transparent objects and subsurface imaging through diffusers. Our low cost prototype may lead to new insights regarding forward and inverse problems in light transport.

/pdf/coded-time-of-flight-cameras-sparse-deconvolution-to-address-4ekubzmjgt.pdf

Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles

Transient imaging is an exciting a new imaging modality that can be used to understand light propagation in complex environments, and to capture and analyze scene properties such as the shape of hidden objects or the reflectance properties of surfaces.Unfortunately, research in transient imaging has so far been hindered by the high cost of the required instrumentation, as well as the fragility and difficulty to operate and calibrate devices such as femtosecond lasers and streak cameras.In this paper, we explore the use of photonic mixer devices (PMD), commonly used in inexpensive time-of-flight cameras, as alternative instrumentation for transient imaging. We obtain a sequence of differently modulated images with a PMD sensor, impose a model for local light/object interaction, and use an optimization procedure to infer transient images given the measurements and model. The resulting method produces transient images at a cost several orders of magnitude below existing methods, while simultaneously simplifying and speeding up the capture process.

Low-budget transient imaging using photonic mixer devices

Time-of-flight range cameras acquire a three-dimensional image of a scene simultaneously for all pixels from a single
viewing location. Attempts to use range cameras for metrology applications have been hampered by the multi-path
problem, which causes range distortions when stray light interferes with the range measurement in a given pixel.
Correcting multi-path distortions by post-processing the three-dimensional measurement data has been investigated, but
enjoys limited success because the interference is highly scene dependent. An alternative approach based on separating
the strongest and weaker sources of light returned to each pixel, prior to range decoding, is more successful, but has only
been demonstrated on custom built range cameras, and has not been suitable for general metrology applications. In this
paper we demonstrate an algorithm applied to both the Mesa Imaging SR-4000 and Canesta Inc. XZ-422 Demonstrator
unmodified off-the-shelf range cameras. Additional raw images are acquired and processed using an optimization
approach, rather than relying on the processing provided by the manufacturer, to determine the individual component
returns in each pixel. Substantial improvements in accuracy are observed, especially in the darker regions of the scene.

/pdf/separating-true-range-measurements-from-multi-path-and-4vke21na8h.pdf

Separating true range measurements from multi-path and scattering interference in commercial range cameras

We present two new closed-form methods for mixed pixel/multipath interference separation in AMCW lidar
systems. The mixed pixel/multipath interference problem arises from the violation of a standard range-imaging
assumption that each pixel integrates over only a single, discrete backscattering source. While a numerical
inversion method has previously been proposed, no close-form inverses have previously been posited. The first
new method models reflectivity as a Cauchy distribution over range and uses four measurements at different
modulation frequencies to determine the amplitude, phase and reflectivity distribution of up to two component
returns within each pixel. The second new method uses attenuation ratios to determine the amplitude and phase
of up to two component returns within each pixel. The methods are tested on both simulated and real data and
shown to produce a significant improvement in overall error. While this paper focusses on the AMCW mixed
pixel/multipath interference problem, the algorithms contained herein have applicability to the reconstruction of
a sparse one dimensional signal from an extremely limited number of discrete samples of its Fourier transform.

/pdf/closed-form-inverses-for-the-mixed-pixel-multipath-2p23y2f9mf.pdf

Closed-form inverses for the mixed pixel/multipath interference problem in AMCW lidar

Amplitude modulated continuous wave (AMCW) lidar systems commonly suffer from non-linear phase and amplitude responses due to a number of known factors such as aliasing and multipath inteference. In order to produce useful range and intensity information it is necessary to remove these perturbations from the measurements. We review the known causes of non-linearity, namely aliasing, temporal variation in correlation waveform shape and mixed pixels/multipath inteference. We also introduce other sources of non-linearity, including crosstalk, modulation waveform envelope decay and non-circularly symmetric noise statistics, that have been ignored in the literature. An experimental study is conducted to evaluate techniques for mitigation of non-linearity, and it is found that harmonic cancellation provides a significant improvement in phase and amplitude linearity.

/pdf/understanding-and-ameliorating-non-linear-phase-and-11dgkw8475.pdf

Understanding and ameliorating non-linear phase and amplitude responses in AMCW Lidar

Full-field amplitude modulated continuous wave range imagers commonly suffer from the mixed pixel problem. This problem is caused by the integration of light from multiple sources by a single pixel, particularly around the edges of objects, resulting in erroneous range measurements. In this paper we present a method for identifying the intensity and range of multiple return values within each pixel, using the harmonic content of the heterodyne beat waveform. Systems capable of measurements at less than 90 degree phase shifts can apply these methods. Our paper builds on previous simulation based work and uses real range data. The method involves the application of the Levy-Fullagar algorithm and the use of the cyclic nature of the beat waveform to extract the mean noise power. We show that this method enables the separation of multiple range sources and also decreases overall ranging error by 30% in the single return case. Error in the two return case was found to increase substantially as relative intensity of the return decreased.

/pdf/mixed-pixel-return-separation-for-a-full-field-ranger-2rihksuo3h.pdf

Mixed pixel return separation for a full-field ranger

Amplitude modulated continuous wave (AMCW) lidar systems suffer from significant systematic errors due to mixed pixels and multipath interference. Commercial systems can achieve centimetre precision, however accuracy is typically an order of magnitude worse limiting practical use of these devices. In this chapter the authors address AMCW measurement formation and the causes of mixed pixels and multipath interference. A comprehensive review of the literature is given, from the first reports of mixed pixels in point-scanning AMCW systems, through to the gamut of research over the past two decades into mixed pixels and multipath interference. An overview of a variety of detection and mitigation techniques, including deconvolution based intra-camera scattering reduction, modelling of intra-scene scattering, correlation waveform deconvolution techniques/multifrequency sampling and standard removal approaches, all of which can be applied to range-data from standard commercial cameras is presented. The chapter concludes with comments on future work for better detection and correction of systematic errors in full-field AMCW lidar.

John Peter Godbaz

Papers

Separating true range measurements from multi-path and scattering interference in commercial range cameras

Closed-form inverses for the mixed pixel/multipath interference problem in AMCW lidar

Understanding and ameliorating non-linear phase and amplitude responses in AMCW Lidar

Mixed pixel return separation for a full-field ranger

Understanding and Ameliorating Mixed Pixels and Multipath Interference in AMCW Lidar