This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

/pdf/distinctive-image-features-from-scale-invariant-keypoints-3waurjqzke.pdf

Distinctive Image Features from Scale-Invariant Keypoints

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on ImageNet classification has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called "perceptual losses"? What elements are critical for their success? To answer these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by large margins on our dataset. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

On robust estimation of the location parameter

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

We present TRex, a flexible and robust Tomographic Reconstruction framework using proximal algorithms. We provide an overview and perform an experimental comparison between the famous iterative reconstruction methods in terms of reconstruction quality in sparse view situations. We then derive the proximal operators for the four best methods. We show the flexibility of our framework by deriving solvers for two noise models: Gaussian and Poisson; and by plugging in three powerful regularizers. We compare our framework to state of the art methods, and show superior quality on both synthetic and real datasets.

TRex: A Tomography Reconstruction Proximal Framework for Robust Sparse View X-Ray Applications.

Defocus imaging techniques, involving the capture and reconstruction of purposely out-of-focus images, have
recently become feasible due to advances in deconvolution methods. This paper evaluates the feasibility of
defocus imaging as a means of increasing the effective dynamic range of conventional image sensors. Blurring
operations spread the energy of each pixel over the surrounding neighborhood; bright regions transfer energy to
nearby dark regions, reducing dynamic range. However, there is a trade-off between image quality and dynamic
range inherent in all conventional sensors.
The approach involves optically blurring the captured image by turning the lens out of focus, modifying that
blurred image with a filter inserted into the optical path, then recovering the desired image by deconvolution.
We analyze the properties of the setup to determine whether any combination can produce a dynamic range
reduction with acceptable image quality. Our analysis considers both properties of the filter to measure local
contrast reduction, as well as the distribution of image intensity at different scales as a measure of global contrast
reduction. Our results show that while combining state-of-the-art aperture filters and deconvolution methods
can reduce the dynamic range of the defocused image, providing higher image quality than previous methods,
rarely does the loss in image fidelity justify the improvements in dynamic range.

/pdf/defocus-techniques-for-camera-dynamic-range-expansion-37sj9z23zy.pdf

Defocus Techniques For Camera Dynamic Range Expansion

This paper discusses a method to reconstruct a transparent ow surface from single camera shot with the aid of a Micro-lens array. An intentionally prepared high frequency background which is placed behind the refractive flow is captured and a curl-free optical flow algorithm is applied between pairs of images taken by different micro-lenses. The computed raw optical ow vector is a blend of motion parallax and background deformation vector due to the underlying flow. Subtracting the motion parallax, which is obtained by calibration, from the total op- optical flow vector yields the background deformation vector. The deflection vectors on each images are used to reconstruct the flow profile. A synthetic data set of fuel injection was used to evaluate the accuracy of the proposed algorithm and good agreement was achieved between the test and reconstructed data. Finally, real light field data of hot air created by a lighter flame is used to reconstruct and show a hot air plume surface.

Light field optical flow for refractive surface reconstruction

Die Erfindung betrifft eine Wand mit oder aus einer auf der Ausenseite davor etwa parallel befestigten, den Sonnenstrahlen ausgesetzten Schicht, insbesondere einervorgehangten Fassade zur Absorption der Sonnenstrahlungsenergie. Die Schicht weist lichtdurchlassiges, warmedammendes Material (3) insbesondere Schaum auf, und zwischen Materialschicht und Wand (1) und/oder innerhalb der Materialschicht ist mindestens eine Sonnenstrahlung absorbierende Flache (2) angeordnet.

Wand zur Absorption der Sonnenstrahlen

Many researchers have been arguing that geometry, bump maps, and BRDFs present a hierarchy of detail that should be exploited for efficient rendering purposes. In practice however, this is often not possible due to inconsistencies in the illumination for these different levels of detail. For example, while bump map rendering often only considers direct illumination and no shadows, geometry-based rendering and BRDFs will mostly also respect shadowing effects, and in many cases even indirect illumination caused by scattered light.In this paper, we present an approach for overcoming these inconsistencies. We introduce an inexpensive method for consistently illuminating height fields and bump maps, as well as simulating BRDFs based on precomputed visibility information. With this information we can achieve a consistent illumination across the levels of detail.The method we propose offers significant performance benefits over existing algorithms for computing the light scattering in height fields and for computing a sampled BRDF representation using a virtual gonioreflectometer. The performance can be further improved by utilizing graphics hardware, which then also allows for interactive display.Finally, our method also approximates the changes in illumination when the height field, bump map, or BRDF is applied to a surface with a different curvature.

Wolfgang Heidrich

Papers

TRex: A Tomography Reconstruction Proximal Framework for Robust Sparse View X-Ray Applications.

Defocus Techniques For Camera Dynamic Range Expansion

Light field optical flow for refractive surface reconstruction

Wand zur Absorption der Sonnenstrahlen

Illuminating Micro Geometry Based on Precomputed Visibility