Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

This paper discusses a new method for capturing the complete appearance of both synthetic and real world objects and scenes, representing this information, and then using this representation to render images of the object from new camera positions. Unlike the shape capture process traditionally used in computer vision and the rendering process traditionally used in computer graphics, our approach does not rely on geometric representations. Instead we sample and reconstruct a 4D function, which we call a Lumigraph. The Lumigraph is a subset of the complete plenoptic function that describes the flow of light at all positions in all directions. With the Lumigraph, new images of the object can be generated very quickly, independent of the geometric or illumination complexity of the scene or object. The paper discusses a complete working system including the capture of samples, the construction of the Lumigraph, and the subsequent rendering of images from this new representation.

The lumigraph

We present a method of recovering high dynamic range radiance maps from photographs taken with conventional imaging equipment. In our method, multiple photographs of the scene are taken with different amounts of exposure. Our algorithm uses these differently exposed photographs to recover the response function of the imaging process, up to factor of scale, using the assumption of reciprocity. With the known response function, the algorithm can fuse the multiple photographs into a single, high dynamic range radiance map whose pixel values are proportional to the true radiance values in the scene. We demonstrate our method on images acquired with both photochemical and digital imaging processes. We discuss how this work is applicable in many areas of computer graphics involving digitized photographs, including image-based modeling, image compositing, and image processing. Lastly, we demonstrate a few applications of having high dynamic range radiance maps, such as synthesizing realistic motion blur and simulating the response of the human visual system.

/pdf/recovering-high-dynamic-range-radiance-maps-from-photographs-zk2yr4fcgd.pdf

Recovering high dynamic range radiance maps from photographs

We present a new approach for modeling and rendering existing architectural scenes from a sparse set of still photographs. Our modeling approach, which combines both geometry-based and imagebased techniques, has two components. The first component is a photogrammetricmodeling method which facilitates the recovery of the basic geometry of the photographed scene. Our photogrammetric modeling approach is effective, convenient, and robust because it exploits the constraints that are characteristic of architectural scenes. The second component is a model-based stereo algorithm, which recovers how the real scene deviates from the basic model. By making use of the model, our stereo technique robustly recovers accurate depth from widely-spaced image pairs. Consequently, our approach can model large architectural environments with far fewer photographs than current image-based modeling approaches. For producing renderings, we present view-dependent texture mapping, a method of compositing multiple views of a scene that better simulates geometric detail on basic models. Our approach can be used to recover models for use in either geometry-based or image-based rendering systems. We present results that demonstrate our approach’s ability to create realistic renderings of architectural scenes from viewpoints far from the original photographs. CR Descriptors: I.2.10 [Artificial Intelligence]: Vision and Scene Understanding Modeling and recovery of physical attributes; I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism Color, shading, shadowing, and texture I.4.8 [Image Processing]: Scene Analysis Stereo; J.6 [Computer-Aided Engineering]: Computer-aided design (CAD).

/pdf/modeling-and-rendering-architecture-from-photographs-a-1wvpl6aaqb.pdf

Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach

Systems and methods for navigating hypermedia using multiple coordinated input/output device sets. Disclosed systems and methods allow a user and/or an author to control what resources are presented on which device sets (whether they are integrated or not), and provide for coordinating browsing activities to enable such a user interface to be employed across multiple independent systems. Disclosed systems and methods also support new and enriched aspects and applications of hypermedia browsing and related business activities.

Method and apparatus for browsing using multiple coordinated device sets

Problems of signal processing arise in image synthesis because of transformations between continuous and discrete representations of 2D images. Aliasing introduced by sampling has received much attention in graphics, but reconstruction of samples into a continuous representation can also cause aliasing as well as other defects in image quality. The problem of designing a filter for use on images is discussed, and a new family of piecewise cubic filters are investigated as a practical demonstration. Two interesting cubic filters are found, one having good antialiasing properties and the other having good image-quality properties. It is also shown that reconstruction using derivative as well as amplitude values can greatly reduce aliasing.

/pdf/reconstruction-filters-in-computer-graphics-4pjy8sduzg.pdf

Reconstruction filters in computer-graphics

Ray tracing produces point samples of an image from a 3-D model Constructing an antialiased digital picture from point samples is difficult without resorting to extremely high sampling densities This paper describes a program that focuses on that problem While it is impossible to eliminate aliasing totally, it has been shown that nonuniform sampling yields aliasing that is less conspicuous to the observer An algorithm is presented for fast generation of nonuniform sampling patterns that are optimal in some sense Some regions of an image may require extra sampling to avoid strong aliasing Deciding where to do extra sampling can be guided by knowledge of how the eye perceives noise as a function of contrast and color Finally, to generate the digital picture, the image must be reconstructed from the samples and resampled at the display pixel rate The nonuniformity of the samples complicates this process, and a new nonuniform reconstruction filter is presented which solves this problem efficiently

/pdf/generating-antialiased-images-at-low-sampling-densities-2d8p1m0n3a.pdf

Generating antialiased images at low sampling densities

Most recent rendering research has concentrated on two subproblems: modeling the reflection of light from materials, and calculating the direct and indirect illumination from light sources and other surfaces. Another key component of a rendering system is the camera model. Unfortunately, current camera models are not geometrically or radiometrically correct and thus are not sufficient for synthesizing images from physically-based rendering programs. In this paper we describe a physically-based camera model for computer graphics. More precisely, a physically-based camera model accurately computes the irradiance on the film given the incoming radiance from the scene. In our model a camera is described as a lens system and film backplane. The lens system consists of a sequence of simple lens elements, stops and apertures. The camera simulation module computes the irradiance on the backplane from the scene radiances using distributed ray tracing. This is accomplished by a detailed simulation of the geometry of ray paths through the lens system, and by sampling the lens system such that the radiometry is computed accurately and efficiently. Because even the most complicated lenses have a relatively small number of elements, the simulation only increases the total rendering time slightly. CR

/pdf/a-realistic-camera-model-for-computer-graphics-54yczrnfws.pdf

A realistic camera model for computer graphics

Nonuniform sampling of images is a useful technique in computer graphics, because a properly designed pattern of samples can make aliasing take the form of high-frequency random noise. In this paper, the technique of nonuniform sampling is extended from two dimensions to include the extra parameter dimensions of distribution ray tracing. A condition for optimality is suggested, and algorithms for approximating optimal sampling are developed. The technique is demonstrated at low sampling densities, so the characteristics of aliasing noise are clearly visible. At supersampling rates, this technique should move noise into frequencies above the passband of the pixel-reconstruction filter.

/pdf/spectrally-optimal-sampling-for-distribution-ray-tracing-juaho1dqkr.pdf

Spectrally optimal sampling for distribution ray tracing

Techniques for improving the resolution of images (either analog images, analytic images, or images having a higher resolution than that of a display device) to be rendered on patterned displays (Fig.10). In one aspect of the present invention, discrete values of a color scan line such as the color scan line (1510) for the color red consist of N samples per emitter (for example, two or more samples (1515r), per emitter Fig. 15)). The samples (1515r) are filtered (e.g. averaged) in order to generate an oversampled color scan line (1520r) (such as a three times oversampled color scan line (Fig. 15)). The new samples of the oversampled color scan line (1520r) are filtered again with box filters (1525) to generate color values (1530) associated with sub-pixel components. The filters (1525) are centered at locations that correspond to the centers of the sub-pixel elements. Thus, the filters (1525) are offset, and consequently operate at distinct positions within the image for each of the color components. Once all of the colors are processed, filter output may be gamma correct (1540) based on the gamma response of a display on which the image is to be rendered.

Donald P. Mitchell

Papers

Reconstruction filters in computer-graphics

Generating antialiased images at low sampling densities

A realistic camera model for computer graphics

Spectrally optimal sampling for distribution ray tracing

Methods apparatus and data structures for enhancing the resolution of images to be rendered on patterned display devices