This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

/pdf/distinctive-image-features-from-scale-invariant-keypoints-3waurjqzke.pdf

Distinctive Image Features from Scale-Invariant Keypoints

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on ImageNet classification has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called "perceptual losses"? What elements are critical for their success? To answer these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by large margins on our dataset. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

On robust estimation of the location parameter

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

We propose a lens design ray tracing engine that is derivative-aware, using automatic differentiation. This derivative-aware property enables the engine to infer gradients of current design parameters, i.e., how design parameters affect a given error metric (e.g., spot RMS or irradiance values), by back-propagating the derivatives through a computational graph via differentiable ray tracing. Our engine not only enables designers to employ gradient descent and variants for design optimization, but also provides a numerically compatible way to perform back-propagation on both the optical design and the post-processing algorithm (e.g., a neural network), making hardware-software end-to-end designs possible. Examples are demonstrated by freeform designs and joint optics-network optimization for extended-depth-of-field applications.

Lens design optimization by back-propagation

This paper focuses on resolving long-standing limitations of parallax barriers by applying formal optimization
methods. We consider two generalizations of conventional parallax barriers. First, we consider general two-layer
architectures, supporting high-speed temporal variation with arbitrary opacities on each layer. Second,
we consider general multi-layer architectures containing three or more light-attenuating layers. This line of
research has led to two new attenuation-based displays. The High-Rank 3D (HR3D) display contains a stacked
pair of LCD panels; rather than using heuristically-defined parallax barriers, both layers are jointly-optimized
using low-rank light field factorization, resulting in increased brightness, refresh rate, and battery life for mobile
applications. The Layered 3D display extends this approach to multi-layered displays composed of compact
volumes of light-attenuating material. Such volumetric attenuators recreate a 4D light field when illuminated
by a uniform backlight. We further introduce Polarization Fields as an optically-efficient and computationally efficient
extension of Layered 3D to multi-layer LCDs. Together, these projects reveal new generalizations to
parallax barrier concepts, enabled by the application of formal optimization methods to multi-layer attenuation-based
designs in a manner that uniquely leverages the compressive nature of 3D scenes for display applications.

Beyond parallax barriers: applying formal optimization methods to multilayer automultiscopic displays

A method is provided for treating gum base to improve its resistance to oxidation without the need for antioxidants wherein air or oxygen is swept from the gum base by means of an inert gas, and the oxygen-free gum base, in particulate form, is coated with a highly oxygenated substance, such as molten sugar, sorbitol solution or hydrogenated starch hydrolysate which acts as a barrier against oxygen. The gum base produced by the above method and chewing gum containing such gum base are also provided.

High dynamic range imaging and display.

In this paper, we introduce a new compressive display architecture for superresolution image presentation that exploits co-design of the optical device configuration and compressive computation Our display allows for superresolution, HDR, or glasses-free 3D presentation

/pdf/a-compressive-superresolution-display-14gpzagxfn.pdf

A Compressive Superresolution Display

We present a novel method to reconstruct a fluid's 3D density and motion based on just a single sequence of images. This is rendered possible by using powerful physical priors for this strongly under-determined problem. More specifically, we propose a novel strategy to infer density updates strongly coupled to previous and current estimates of the flow motion. Additionally, we employ an accurate discretization and depth-based regularizers to compute stable solutions. Using only one view for the reconstruction reduces the complexity of the capturing setup drastically and could even allow for online video databases or smart-phone videos as inputs. The reconstructed 3D velocity can then be flexibly utilized, e.g., for re-simulation, domain modification or guiding purposes. We will demonstrate the capacity of our method with a series of synthetic test cases and the reconstruction of real smoke plumes captured with a Raspberry Pi camera.

Wolfgang Heidrich

Papers

Lens design optimization by back-propagation

Beyond parallax barriers: applying formal optimization methods to multilayer automultiscopic displays

High dynamic range imaging and display.

A Compressive Superresolution Display

Coupled Fluid Density and Motion from Single Views