Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

Acknowledgements 1. Introduction 2. Electromagnetic wave propagation 3. The absorption of light 4. Specular reflection 5. Single particle scattering: perfect spheres 6. Single particle scattering: irregular particles 7. Propagation in a nonuniform medium: the equation of radiative transfer 8. The bidirectional reflectance of a semi-infinite medium 9. The opposition effect 10. A miscellany of bidirectional reflectances and related quantities 11. Integrated reflectances and planetary photometry 12. Photometric effects of large scale roughness 13. Polarization 14. Reflectance spectroscopy 15. Thermal emission and emittance spectroscopy 16. Simultaneous transport of energy by radiation and conduction Appendix A. A brief review of vector calculus Appendix B. Functions of a complex variable Appendix C. The wave equation in spherical coordinates Appendix D. Fraunhoffer diffraction by a circular hole Appendix E. Table of symbols Bibliography Index.

Theory of Reflectance and Emittance Spectroscopy

Since the first shape-from-shading (SFS) technique was developed by Horn in the early 1970s, many different approaches have emerged. In this paper, six well-known SFS algorithms are implemented and compared. The performance of the algorithms was analyzed on synthetic images using mean and standard deviation of depth (Z) error, mean of surface gradient (p, q) error, and CPU timing. Each algorithm works well for certain images, but performs poorly for others. In general, minimization approaches are more robust, while the other approaches are faster.

https://crcv.ucf.edu/papers/Zhang_Tsai_Cryer_Shah_PAMI1999.pdf

Shape-from-shading: a survey

We prove that the set of all Lambertian reflectance functions (the mapping from surface normals to intensities) obtained with arbitrary distant light sources lies close to a 9D linear subspace. This implies that, in general, the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace, explaining prior empirical results. We also provide a simple analytic characterization of this linear space. We obtain these results by representing lighting using spherical harmonics and describing the effects of Lambertian materials as the analog of a convolution. These results allow us to construct algorithms for object recognition based on linear methods as well as algorithms that use convex optimization to enforce nonnegative lighting functions. We also show a simple way to enforce nonnegative lighting when the images of an object lie near a 4D linear space. We apply these algorithms to perform face recognition by finding the 3D model that best matches a 2D query image.

/pdf/lambertian-reflectance-and-linear-subspaces-k79c4cysdg.pdf

Lambertian reflectance and linear subspaces

In this paper we consider the problem of computing the 3D shape of an unknown, arbitrarily-shaped scene from multiple photographs taken at known but arbitrarily-distributed viewpoints. By studying the equivalence class of all 3D shapes that reproduce the input photographs, we prove the existence of a special member of this class, the photo hull, that (1) can be computed directly from photographs of the scene, and (2) subsumes all other members of this class. We then give a provably-correct algorithm, called Space Carving, for computing this shape and present experimental results on complex real-world scenes. The approach is designed to (1) capture photorealistic shapes that accurately model scene appearance from a wide range of viewpoints, and (2) account for the complex interactions between occlusion, parallax, shading, and their view-dependent effects on scene-appearance.

A Theory of Shape by Space Carving

The directional distribution of radiant flux reflected from roughened surfaces is analyzed on the basis of geometrical optics. The analytical model assumes that the surface consists of small, randomly disposed, mirror-like facets. Specular reflection from these facets plus a diffuse component due to multiple reflections and/or internal scattering are postulated as the basic mechanisms of the reflection process. The effects of shadowing and masking of facets by adjacent facets are included in the analysis. The angular distributions of reflected flux predicted by the analysis are in very good agreement with experiment for both metallic and nonmetallic surfaces. Moreover, the analysis successfully predicts the off-specular maxima in the reflection distribution which are observed experimentally and which emerge as the incidence angle increases. The model thus affords a rational explanation for the off-specular peak phenomenon in terms of mutual masking and shadowing of mirror-like, specularly reflecting surface facets.

K. E. Torrance

Papers

Theory for off-specular reflection from roughened surfaces

Theory for Off-Specular Reflection From Roughened Surfaces*

Directional emittance of an electric nonconductor as a function of surface roughness and wavelength