This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

/pdf/distinctive-image-features-from-scale-invariant-keypoints-3waurjqzke.pdf

Distinctive Image Features from Scale-Invariant Keypoints

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on ImageNet classification has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called "perceptual losses"? What elements are critical for their success? To answer these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by large margins on our dataset. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

On robust estimation of the location parameter

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

We describe a system for transforming an off-the-shelf atbed scanner into a $200 scan backend for large format cameras. While we describe both software and hardware aspects, the focus of the paper is on software issues such as color calibration and removal of scanner artifacts. With current scanner technology, the resulting camera system is capable of taking black&white, color, or near-infrared photographs with up to 490 million pixels. Our analysis shows that we achieve actual optical resolutions close to the theoretical maximum, and that color reproduction is comparable to commercial camera systems. We believe that the camera system described here has many potential applications in image-based modeling and rendering, cultural heritage projects, and professional digital photography.

The Design of an Inexpensive Very High Resolution Scan Camera System

Feathers, unlike other cutaneous appendages such as hair, fur, or scales have a definite structure. Variation in feather structure creates a wide range of resulting appearances. Collectively, feather structure determines the appearance of the feather coat, which can largely affect the resulting look of a feathered object (bird). In this paper we define the structure of individual feathers using a parameterization based on biological structure and substructures of actual feathers. We show that our parameterization can generate a large variety of feathers at multiple levels of detail and provide an initial step to semi-automatically generating a wide range of feather coats. This is achieved by specifying an intuitive interpolation between different structures and ages of feathers.

A Biologically-Parameterized Feather Model

Real-time procedural shading was once seen as a distant dream. When the first version of this course was offered four years ago, real-time shading was possible, but only with one-of-a-kind hardware or by combining the effects of tens to hundreds of rendering passes. Today, almost every new computer comes with graphics hardware capable of interactively executing shaders of thousands to tens of thousands of instructions. This course has been redesigned to address today's real-time shading capabilities and to provide more practical information for practitioners. The morning sessions cover the more advanced technical aspects of creating a shading system. The afternoon sessions cover practical details of real-time shading use, including an overview of recently developed algorithms that run well on today's shading hardware and presentations on the latest hardware developments from several leading hardware vendors. The course concludes with the popular panel-style question and answer session, where participants can ask questions of any presenter or suggest topics of discussion.

Real-Time Shading

Many natural objects, and general layered materials, have non-linear reflection behaviour along wavelengths. An accurate representation of phenomena such as interference and colour separation generally requires a fine spectral representation of light instead of the commonly used RGB components. In this article, we introduce and experiment with a general approach to create similar and convincing effects, with a simple RGB BRDF model. We present a model for both specular and diffuse reflection.

/pdf/a-simple-layered-rgb-brdf-model-3spojpfzq5.pdf

A simple layered RGB BRDF model

Countershading is a common technique for local image contrast manipulations, and is widely used both in automatic settings, such as image sharpening and tonemapping, as well as under artistic control, such as in paintings and interactive image processing software. Unfortunately, countershading is a double-edged sword: while correctly chosen parameters for a given viewing condition can significantly improve the image sharpness or trick the human visual system into perceiving a higher contrast than physically present in an image, wrong parameters, or different viewing conditions can result in objectionable halo artifacts. In this paper we investigate the perception of countershading in the context of a novel mask-based contrast enhancement algorithm and analyze the circumstances under which the resulting profiles turn from image enhancement to artifact for a range of parameters and viewing conditions. Our experimental results can be modeled as a function of the width of the countershading profile. We employ this empirical function in a range of applications such as image resizing, view dependent tone mapping, and countershading analysis in photographs and works of fine art. © 2012 Wiley Periodicals, Inc.

/pdf/unsharp-masking-countershading-and-halos-enhancements-or-3r9va1feos.pdf

Wolfgang Heidrich

Papers

The Design of an Inexpensive Very High Resolution Scan Camera System

A Biologically-Parameterized Feather Model

Real-Time Shading

A simple layered RGB BRDF model

Unsharp Masking, Countershading and Halos: Enhancements or Artifacts?