This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

/pdf/distinctive-image-features-from-scale-invariant-keypoints-3waurjqzke.pdf

Distinctive Image Features from Scale-Invariant Keypoints

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on ImageNet classification has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called "perceptual losses"? What elements are critical for their success? To answer these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by large margins on our dataset. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

On robust estimation of the location parameter

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

Without specialized sensor technology or custom, multi-chip cameras, high dynamic range imaging typically involves time-sequential capture of multiple photographs. The obvious downside to this approach is that it cannot easily be applied to images with moving objects, especially if the motions are complex. In this paper, we take a novel view of HDR capture, which is based on a computational photography approach. We propose to first optically encode both the low dynamic range portion of the scene and highlight information into a low dynamic range image that can be captured with a conventional image sensor. This step is achieved using a cross-screen, or star filter. Second, we decode, in software, both the low dynamic range image and the highlight information. Lastly, these two portions can be combined to form an image of a higher dynamic range than the regular sensor dynamic range.

/pdf/glare-encoding-of-high-dynamic-range-images-of60efxbc2.pdf

Glare encoding of high dynamic range images

Multiperspective images generated from a collection of photographs or a videostream can be used to effectively summarize long, roughly planar scenes such as city streets. The final image will span a larger field of view than any single input image. However, common projections used to make these images, including cross-slits and pushbroom projections, may suffer from depth-related distortions in non-planar scenes. In this paper, we use an aspect-ratio distortion metric to compare these images to standard perspective projections. By minimizing this error metric we can automatically define the picture surface and viewpoints of a multiperspective image that reduces distortion artifacts. This optimization requires only a coarse estimate of scene geometry which can be provided as a depth map or a 2D spatial importance map defining interesting parts of the scene. These maps can be automatically constructed in most cases, allowing rapid generation of images of very long scenes.

Automatic Multiperspective Images

After decades of research on digital representations of material and object appearance, computer graphics has more recently turned to the problem of creating physical artifacts with controllable appearance characteristics. While this work has mostly progressed in two parallel streams – display technologies as well as novel fabrication processes – we believe there is a large overlap and the potential for synergies between these two approaches. In this report, we summarize research efforts from the worlds of fabrication display, and categorize the different approaches into a common taxonomy. We believe that this report can serve as a basis for systematic exploration of the design space in future research.

http://www.matthias.hullin.net/publications/HullinEtAl-STARComputationalFabricationDisplay-EG2013.pdf

Computational Fabrication and Display of Material Appearance

Imaging systems have long been designed in separated steps: experience-driven optical design followed by sophisticated image processing. Although recent advances in computational imaging aim to bridge the gap in an end-to-end fashion, the image formation models used in these approaches have been quite simplistic, built either on simple wave optics models such as Fourier transform, or on similar paraxial models. Such models only support the optimization of a single lens surface, which limits the achievable image quality. To overcome these challenges, we propose a general end-to-end complex lens design framework enabled by a differentiable ray tracing image formation model. Specifically, our model relies on the differentiable ray tracing rendering engine to render optical images in the full field by taking into account all on/off-axis aberrations governed by the theory of geometric optics. Our design pipeline can jointly optimize the lens module and the image reconstruction network for a specific imaging task. We demonstrate the effectiveness of the proposed method on two typical applications, including large field-of-view imaging and extended depth-of-field imaging. Both simulation and experimental results show superior image quality compared with conventional lens designs. Our framework offers a competitive alternative for the design of modern imaging systems.

/pdf/end-to-end-complex-lens-design-with-differentiate-ray-4xdiaxveqa.pdf

End-to-end complex lens design with differentiate ray tracing

contrast ratio and amplitude resolution are rapidly growing display specifications. Through a series of human factor studies we have developed simple guidelines for these specifications including viewer preference for luminance, optimal contrast ratio and amplitude resolution under realistic conditions. 1. Introduction past, conventional displays have been largely limited to a dynamic range similar to paper under office lighting conditions - approximately two or three orders of magnitude starting at a grayish black and finishing in the hundreds of cd/m 2 . This paradigm of hundreds-to-one contrast ratio, limited luminance and an amplitude resolution in the hundreds of steps is shifting today. Novel display technologies are emerging with the potential of much higher contrast and brightness. Moreover, even existing display technology is being pushed to the limit with a strong increase in display performance. This trend proceeds unevenly with contrast ratio rising faster than luminance, and amplitude resolution remaining largely static. As a result, many display designs make sub-optimal use of the device capabilities. This paper presents a series of human factor studies that aim to provide a basic framework of luminance, contrast ratio and amplitude resolution and their interaction. The use of a High Dynamic Range (HDR) display (1) as the imaging tool for the study, allows a large enough range for each variable to encompass all current and most near-future display technologies. The results of the study can be used to make design decisions for future displays as well as more realistic comparisons of existing devices. 2. Background Video Electronics Standards Association (VESA) and International Organization for Standardization (ISO) provide common guidelines for display specification including luminance and contrast ratio. Peak luminance is generally easy to measure and reported relatively accurately by the industry. Contrast ratio is significantly more challenging. A proper contrast ratio

Wolfgang Heidrich

Papers

Glare encoding of high dynamic range images

Automatic Multiperspective Images

Computational Fabrication and Display of Material Appearance

End-to-end complex lens design with differentiate ray tracing

25.3: Observations of Luminance, Contrast and Amplitude Resolution of Displays