From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

The recovery of objects obscured by scattering is an important goal in imaging and has been approached by exploiting, for example, coherence properties, ballistic photons or penetrating wavelengths. Common methods use scattered light transmitted through an occluding material, although these fail if the occluder is opaque. Light is scattered not only by transmission through objects, but also by multiple reflection from diffuse surfaces in a scene. This reflected light contains information about the scene that becomes mixed by the diffuse reflections before reaching the image sensor. This mixing is difficult to decode using traditional cameras. Here we report the combination of a time-of-flight technique and computational reconstruction algorithms to untangle image information mixed by diffuse reflection. We demonstrate a three-dimensional range camera able to look around a corner using diffusely reflected light that achieves sub-millimetre depth precision and centimetre lateral precision over 40 cm×40 cm×40 cm of hidden space. An important goal in optics is to image objects hidden by turbid media, although line-of-sight techniques fail when the obscuring medium becomes opaque. Veltenet al. use ultrafast imaging techniques to recover three-dimensional shapes of non-line-of-sight objects after reflection from diffuse surfaces.

/pdf/recovering-three-dimensional-shape-around-a-corner-using-3j2ptae4fl.pdf

Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging.

Since their inception in the 1930–1960s, the research disciplines of computational imaging and machine learning have followed parallel tracks and, during the last two decades, experienced explosive growth drawing on similar progress in mathematical optimization and computing hardware. While these developments have always been to the benefit of image interpretation and machine vision, only recently has it become evident that machine learning architectures, and deep neural networks in particular, can be effective for computational image formation, aside from interpretation. The deep learning approach has proven to be especially attractive when the measurement is noisy and the measurement operator ill posed or uncertain. Examples reviewed here are: super-resolution; lensless retrieval of phase and complex amplitude from intensity; photon-limited scenes, including ghost imaging; and imaging through scatter. In this paper, we cast these works in a common framework. We relate the deep-learning-inspired solutions to the original computational imaging formulation and use the relationship to derive design insights, principles, and caveats of more general applicability. We also explore how the machine learning process is aided by the physics of imaging when ill posedness and uncertainties become particularly severe. It is hoped that the present unifying exposition will stimulate further progress in this promising field of research.

On the use of deep learning for computational imaging

A three-dimensional imaging system which distributes the optical illumination over the full field-of-view is sought after. Here, the authors demonstrate the capability of reconstructing 128 × 128 pixel resolution three-dimensional scenes to an accuracy of 3 mm as well as real-time video with a frame-rate up to 12 Hz.

Single-pixel three-dimensional imaging with time-based depth resolution

We present RF-Capture, a system that captures the human figure -- i.e., a coarse skeleton -- through a wall. RF-Capture tracks the 3D positions of a person's limbs and body parts even when the person is fully occluded from its sensor, and does so without placing any markers on the subject's body. In designing RF-Capture, we built on recent advances in wireless research, which have shown that certain radio frequency (RF) signals can traverse walls and reflect off the human body, allowing for the detection of human motion through walls. In contrast to these past systems which abstract the entire human body as a single point and find the overall location of that point through walls, we show how we can reconstruct various human body parts and stitch them together to capture the human figure. We built a prototype of RF-Capture and tested it on 15 subjects. Our results show that the system can capture a representative human figure through walls and use it to distinguish between various users.

/pdf/capturing-the-human-figure-through-a-wall-yhtcbqzs16.pdf

Capturing the human figure through a wall

Imagers that use their own illumination can capture three-dimensional (3D) structure and reflectivity information. With photon-counting detectors, images can be acquired at extremely low photon fluxes. To suppress the Poisson noise inherent in low-flux operation, such imagers typically require hundreds of detected photons per pixel for accurate range and reflectivity determination. We introduce a low-flux imaging technique, called first-photon imaging, which is a computational imager that exploits spatial correlations found in real-world scenes and the physics of low-flux measurements. Our technique recovers 3D structure and reflectivity from the first detected photon at each pixel. We demonstrate simultaneous acquisition of sub-pulse duration range and 4-bit reflectivity information in the presence of high background noise. First-photon imaging may be of considerable value to both microscopy and remote sensing.

https://science.sciencemag.org/content/sci/343/6166/58.full.pdf

First-Photon Imaging

We show that multi-path analysis using images from a timeof-flight (ToF) camera provides a tantalizing opportunity to infer about 3D geometry of not only visible but hidden parts of a scene. We provide a novel framework for reconstructing scene geometry from a single viewpoint using a camera that captures a 3D time-image I(x, y, t) for each pixel. We propose a framework that uses the time-image and transient reasoning to expose scene properties that may be beyond the reach of traditional computer vision. We corroborate our theory with free space hardware experiments using a femtosecond laser and an ultrafast photo detector array. The ability to compute the geometry of hidden elements, unobservable by both the camera and illumination source, will create a range of new computer vision opportunities.

/pdf/looking-around-the-corner-using-transient-imaging-4umcfnuqnm.pdf

Looking around the corner using transient imaging

Capturing depth and reflectivity images at low light levels from active illumination of a scene has wide-ranging applications. Conventionally, even with detectors sensitive to individual photons, hundreds of photon detections are needed at each pixel to mitigate Poisson noise. We develop a robust method for estimating depth and reflectivity using fixed dwell time per pixel and on the order of one detected photon per pixel averaged over the scene. Our computational image formation method combines physically accurate single-photon counting statistics with exploitation of the spatial correlations present in real-world reflectivity and 3-D structure. Experiments conducted in the presence of strong background light demonstrate that our method is able to accurately recover scene depth and reflectivity, while traditional imaging methods based on maximum likelihood (ML) estimation or approximations thereof lead to noisier images. For depth, performance compares favorably to signal-independent noise removal algorithms such as median filtering or block-matching and 3-D filtering (BM3D) applied to the pixelwise ML estimate; for reflectivity, performance is similar to signal-dependent noise removal algorithms such as Poisson nonlocal sparse PCA and BM3D with variance-stabilizing transformation. Our framework increases photon efficiency 100-fold over traditional processing and also improves, somewhat, upon first-photon imaging under a total acquisition time constraint in raster-scanned operation. Thus, our new imager will be useful for rapid, low-power, and noise-tolerant active optical imaging, and its fixed dwell time will facilitate parallelization through use of a detector array.

Photon-Efficient Computational 3-D and Reflectivity Imaging With Single-Photon Detectors

Capturing depth and reflectivity images at low light levels from active illumination of a scene has wide-ranging applications. Conventionally, even with single-photon detectors, hundreds of photon detections are needed at each pixel to mitigate Poisson noise. We develop a robust method for estimating depth and reflectivity using on the order of 1 detected photon per pixel averaged over the scene. Our computational imager combines physically accurate single-photon counting statistics with exploitation of the spatial correlations present in real-world reflectivity and 3D structure. Experiments conducted in the presence of strong background light demonstrate that our computational imager is able to accurately recover scene depth and reflectivity, while traditional maximum-likelihood based imaging methods lead to estimates that are highly noisy. Our framework increases photon efficiency 100-fold over traditional processing and also improves, somewhat, upon first-photon imaging under a total acquisition time constraint in raster-scanned operation. Thus our new imager will be useful for rapid, low-power, and noise-tolerant active optical imaging, and its fixed dwell time will facilitate parallelization through use of a detector array.

Photon-Efficient Computational 3D and Reflectivity Imaging with Single-Photon Detectors

We propose a framework for simultaneous phase unwrapping and multipath interference cancellation (SPUMIC) in homodyne time-of-flight (ToF) cameras. Our multi-frequency acquisition framework is based on parametric modeling of the multipath interference phenomena. We use robust spectral estimation methods with low computational complexity to detect and estimate multipath parameters. Using simulations and analysis we demonstrate that our proposed solution is implementable in real-time on existing ToF cameras without requiring any hardware modifications.

https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/06607553.pdf

Ahmed Kirmani

Papers

First-Photon Imaging

Looking around the corner using transient imaging

Photon-Efficient Computational 3-D and Reflectivity Imaging With Single-Photon Detectors

Photon-Efficient Computational 3D and Reflectivity Imaging with Single-Photon Detectors

SPUMIC: Simultaneous phase unwrapping and multipath interference cancellation in time-of-flight cameras using spectral methods