This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

/pdf/distinctive-image-features-from-scale-invariant-keypoints-3waurjqzke.pdf

Distinctive Image Features from Scale-Invariant Keypoints

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on ImageNet classification has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called "perceptual losses"? What elements are critical for their success? To answer these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by large margins on our dataset. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

On robust estimation of the location parameter

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

We introduce a new approach to capturing refraction in transparent media, which we call light field background oriented Schlieren photography. By optically coding the locations and directions of light rays emerging from a light field probe, we can capture changes of the refractive index field between the probe and a camera or an observer. Our prototype capture setup consists of inexpensive off-the-shelf hardware, including inkjet-printed transparencies, lenslet arrays, and a conventional camera. By carefully encoding the color and intensity variations of 4D light field probes, we show how to code both spatial and angular information of refractive phenomena. Such coding schemes are demonstrated to allow for a new, single image approach to reconstructing transparent surfaces, such as thin solids or surfaces of fluids. The captured visual information is used to reconstruct refractive surface normals and a sparse set of control points independently from a single photograph.

/pdf/computational-schlieren-photography-with-light-field-probes-4ealybs60b.pdf

Computational Schlieren Photography with Light Field Probes

In order to clean gases containing noxious components by contacting the gases with a solid material layer adhering to carrier bodies, the solid material layer is applied to the carrier body by wetting with an impregnating solution and then drying the solution on the carrier bodies. After being used up by reaction with the gases to be cleaned, the solid material layer is removed from the carrier bodies, after which it can be replaced by wetting with impregnating solution and drying again. The carrier bodies are stationary and the impregnating solution is trickled over them. Once the carrier bodies are wetted, irrigation with the impregnation solution is interrupted and the solution is dried on the carrier bodies. When the gas reaction at the solid material layer thus formed weakens, the stationary carrier bodies are re-irrigated with impregnation solution and dried. After many repeated impregnations the solid material layer is washed away by flushing or flooding the carrier bodies with a wash solution.

Process and apparatus for cleaning hot gases

Sampling of complex direct illumination in the form of high dynamic range (HDR) environment maps has recieved a lot of attention in computer graphics, with major applications in realistic relighting. The best known techniques for direct illumination sample from the product of the incident illumination and the surface reflectance [ Burke et al. 2005 ]. Recent work on sampling from HDR video sequences only take the dynamic importance of the illumination into account while proposing samples for the video sequence. This can be sub-optimal in the presence of high frequencies in both the illumination as well as the surface BRDFs. In this work, we aim to efficiently sample from the product distribution of the illumination and the BRDF in a video sequence with dynamic illumination using asequential Monte Carlo (SMC) sampling strategy. The basic idea is to generate samples according to the product distribution in the first frame of the sequence, and thereafter to filter these samples (particles) in the subsequent frames according to the dynamic product distribution. This sequential sampling mechanism is more efficient than independently sampling from the product distribution at each time step (Figure 1), especially for scenes with high frequencies in both the dynamic illumination and the BRDF. We employ bidirectional importance sampling for generating samples in the first frame of the sequence.

Sequential sampling for dynamic environment maps

Proceedings of the 17th Eurographics conference on Rendering Techniques

NRC’s NEW-EARTH Lab has demonstrated in the laboratory a Self-Coherent Camera (SCC) concept combined with a Tilt-Gaussian-Vortex focal plane mask (FPM). This speckle suppression technique, a.k.a. Fast Atmospheric SCC Technique (FAST), can enhance the contrast up to 100 times. Based on this success, NRC is now building SPIDERS, a visitor instrument for Subaru telescope to be installed on the infrared Nasmyth platform behind AO188 and the new Subaru Beam Switcher. The beam can be either shared between SPIDERS and SCExAO for simultaneous observations or sent entirely to only one instrument. SPIDERS should also benefit from the upcoming AO188 deformable mirror (DM) upgrade (64x64 actuators) turning A188 to AO3k. The key-components of SPIDERS are an ALPAO DM468, used as a second-stage AO correction, a pupil apodizer mask, a Tilt-Gaussian FPM, a Lyot stop, a beam-splitter feeding (i), a C-RED2 camera imaging a 5” FoV in narrow bands and (ii), an imaging Fourier-Transform Spectrograph and a SAPHIRA camera for spectroscopy up to R~20,000 over a 3.3” FoV. SPIDERS optical design is fully reflective up to the FPM to avoid chromatic aberrations and reduce the number of surfaces. Two off-axis ellipsoid mirrors are enough to form the pupil planes required on the DM and the apodizer mask, and the f/64 focus on the FPM. Only lenses are used from the FPM up to the C-RED2 camera to mitigate the sensitivity of the SCC to vibrations. The Lyot stop reflects the blocked light to a camera acting as a Low-Order Wavefront Sensor complementing the SCC focal plane wavefront sensing.

/pdf/optical-design-of-spiders-a-subaru-pathfinder-instrument-for-j3ziuso0.pdf

Wolfgang Heidrich

Papers

Computational Schlieren Photography with Light Field Probes

Process and apparatus for cleaning hot gases

Sequential sampling for dynamic environment maps

Proceedings of the 17th Eurographics conference on Rendering Techniques

Optical design of SPIDERS, a Subaru Pathfinder Instrument for Detecting Exoplanets and Retrieving Spectra