Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

We present a neural network-based upright frontal face detection system. A retinally connected neural network examines small windows of an image and decides whether each window contains a face. The system arbitrates between multiple networks to improve performance over a single network. We present a straightforward procedure for aligning positive face examples for training. To collect negative examples, we use a bootstrap algorithm, which adds false detections into the training set as training progresses. This eliminates the difficult task of manually selecting nonface training examples, which must be chosen to span the entire space of nonface images. Simple heuristics, such as using the fact that faces rarely overlap in images, can further improve the accuracy. Comparisons with several other state-of-the-art face detection systems are presented, showing that our system has comparable performance in terms of detection and false-positive rates.

Neural network-based face detection

Cellular automata are used as simple mathematical models to investigate self-organization in statistical mechanics. A detailed analysis is given of "elementary" cellular automata consisting of a sequence of sites with values 0 or 1 on a line, with each site evolving deterministically in discrete time steps according to definite rules involving the values of its nearest neighbors. With simple initial configurations, the cellular automata either tend to homogeneous states, or generate self-similar patterns with fractal dimensions \ensuremath{\simeq} 1.59 or \ensuremath{\simeq} 1.69. With "random" initial configurations, the irreversible character of the cellular automaton evolution leads to several self-organization phenomena. Statistical properties of the structures generated are found to lie in two universality classes, independent of the details of the initial state or the cellular automaton rules. More complicated cellular automata are briefly considered, and connections with dynamical systems theory and the formal theory of computation are discussed.

Statistical mechanics of cellular automata

A freely available dataset produced from three million Landsat satellite images reveals substantial changes in the distribution of global surface water over the past 32 years and their causes, from climate change to human actions. The distribution of surface water has been mapped globally, and local-to-regional studies have tracked changes over time. But to date, there has been no global and methodologically consistent quantification of changes in surface water over time. Jean-Francois Pekel and colleagues have analysed more than three million Landsat images to quantify month-to-month changes in surface water at a resolution of 30 metres and over a 32-year period. They find that surface waters have declined by almost 90,000 square kilometres—largely in the Middle East and Central Asia—but that surface waters equivalent to about twice that area have been created elsewhere. Drought, reservoir creation and water extraction appear to have driven most of the changes in surface water over the past decades. The location and persistence of surface water (inland and coastal) is both affected by climate and human activity1 and affects climate2,3, biological diversity4 and human wellbeing5,6. Global data sets documenting surface water location and seasonality have been produced from inventories and national descriptions7, statistical extrapolation of regional data8 and satellite imagery9,10,11,12, but measuring long-term changes at high resolution remains a challenge. Here, using three million Landsat satellite images13, we quantify changes in global surface water over the past 32 years at 30-metre resolution. We record the months and years when water was present, where occurrence changed and what form changes took in terms of seasonality and persistence. Between 1984 and 2015 permanent surface water has disappeared from an area of almost 90,000 square kilometres, roughly equivalent to that of Lake Superior, though new permanent bodies of surface water covering 184,000 square kilometres have formed elsewhere. All continental regions show a net increase in permanent water, except Oceania, which has a fractional (one per cent) net loss. Much of the increase is from reservoir filling, although climate change14 is also implicated. Loss is more geographically concentrated than gain. Over 70 per cent of global net permanent water loss occurred in the Middle East and Central Asia, linked to drought and human actions including river diversion or damming and unregulated withdrawal15,16. Losses in Australia17 and the USA18 linked to long-term droughts are also evident. This globally consistent, validated data set shows that impacts of climate change and climate oscillations on surface water occurrence can be measured and that evidence can be gathered to show how surface water is altered by human activities. We anticipate that this freely available data will improve the modelling of surface forcing, provide evidence of state and change in wetland ecotones (the transition areas between biomes), and inform water-management decision-making.

High-resolution mapping of global surface water and its long-term changes

The application of volume-rendering techniques to the display of surfaces from sampled scalar functions of three spatial dimensions is discussed. It is not necessary to fit geometric primitives to the sampled data; images are formed by directly shading each sample and projecting it onto the picture plane. Surface-shading calculations are performed at every voxel with local gradient vectors serving as surface normals. In a separate step, surface classification operators are applied to compute a partial opacity of every voxel. Operators that detect isovalue contour surfaces and region boundary surfaces are examined. The technique is simple and fast, yet displays surfaces exhibiting smooth silhouettes and few other aliasing artifacts. The use of selective blurring and supersampling to further improve image quality is described. Examples from molecular graphics and medical imaging are given. >

/pdf/display-of-surfaces-from-volume-data-419bqc0iiy.pdf

Display of surfaces from volume data

Digital control of color television monitors—in particular, via frame buffers—has added precise control of a large subset of human colorspace to the capabilities of computer graphics. This subset is the gamut of colors spanned by the red, green, and blue (RGB) electron guns exciting their respective phosphors. It is called the RGB monitor gamut. Full-blown color theory is a quite complex subject involving physics, psychology, and physiology, but restriction to the RGB monitor gamut simplifies matters substantially. It is linear, for example, and admits to familiar spatial representations. This paper presents a set of alternative models of the RGB monitor gamut based on the perceptual variables hue (H), saturation (S), and value (V) or brightness (L). Algorithms for transforming between these models are derived. Particular emphasis is placed on an RGB to HSV non-trigonometric pair of transforms which have been used successfully for about four years in frame buffer painting programs. These are fast, accurate, and adequate in many applications. Computationally more difficult transform pairs are sometimes necessary, however. Guidelines for choosing among the models are provided. Psychophysical corrections are described within the context of the definitions established by the NTSC (National Television Standards Committee).

/pdf/color-gamut-transform-pairs-4z495rzfnb.pdf

Color gamut transform pairs

A classical problem of imaging—the matting problem—is separation of a non-rectangular foreground image from a (usually) rectangular background image—for example, in a film frame, extraction of an actor from a background scene to allow substitution of a different background. Of the several attacks on this difficult and persistent problem, we discuss here only the special case of separating a desired foreground image from a background of a constant, or almost constant, backing color. This backing color has often been blue, so the problem, and its solution, have been called blue screen matting. However, other backing colors, such as yellow or (increasingly) green, have also been used, so we often generalize to constant color matting. The mathematics of constant color matting is presented and proven to be unsolvable as generally practiced. This, of course, flies in the face of the fact that the technique is commonly used in film and video, so we demonstrate constraints on the general problem that lead to solutions, or at least significantly prune the search space of solutions. We shall also demonstrate that an algorithmic solution is possible by allowing the foreground object to be shot against two constant backing colors—in fact, against two completely arbitrary backings so long as they differ everywhere.

Blue screen matting

Although fractal models of natural phenomena have received much attention recently, there are other models of complex natural objects which have been around longer in Computer Imagery but are not widely known. These are procedural models of plants and trees. An interesting class of these models is presented here which handles plant growth, sports an efficient data representation, and has a high “database amplification” factor. It is based on an extension of the well-known formal languages of symbol strings to the lesser-known formal languages of labeled graphs. It is so tempting to describe these plant models as “fractal” that the similarities of this class of models with fractal models are explored in an attempt at rapprochement. The models are not fractal so the common parts of fractal theory and plant theory are abstracted to form a class of objects, the graftals. This class may prove to be of great interest to the future of Computer Imagery. Determinism is shown to provide adequate complexity, whereas randomness is only convenient and often inefficient. Finally, a nonfractal, nongraftal family of trees by Bill Reeves is introduced to emphasize some of the paper's nongrammatical themes.

Plants, fractals, and formal languages

Currerntly texture mapping onto projections of 3-D surfaces is time consuming and subject to considerable aliasing errors. Usually the procedure is to perform some inverse mapping from the area of the pixel onto the surface texture. It is difficult to do this correctly. There is an alternate approach where the texture surface is transformed as a 2-D image until it conforms to a projection of a polygon placed arbitrarily in 3-space. The great advantage of this approach is that the 2-D transformation can be decomposed into two simple transforms, one in horizontal and the other in vertical scanline order. horizontal scanline order, Sophisticated light calculation is also time consuming and difficult to calculate correctly on projected polygons. Instead of calculating the lighting based on the position of the polygon, lights, and eye, the lights and eye can be transformed to a corresponding position for a unit square which we can consider to be a canonical polygon. After this canonical polygon is correctly textured and shaded it can be easily conformed to the projection of the 3-D surface.

/pdf/3-d-transformations-of-images-in-scanline-order-t6qyt1z4xu.pdf

Alvy Ray Smith

Papers

Color gamut transform pairs

Blue screen matting

Plants, fractals, and formal languages

3-D transformations of images in scanline order

Real-time language recognition by one-dimensional cellular automata