Shape-from-shading: a survey
01 Aug 1999-IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE Computer Society)-Vol. 21, Iss: 8, pp 690-706
TL;DR: Six well-known SFS algorithms are implemented and compared, and the performance of the algorithms was analyzed on synthetic images using mean and standard deviation of depth error, mean of surface gradient error, and CPU timing.
Abstract: Since the first shape-from-shading (SFS) technique was developed by Horn in the early 1970s, many different approaches have emerged. In this paper, six well-known SFS algorithms are implemented and compared. The performance of the algorithms was analyzed on synthetic images using mean and standard deviation of depth (Z) error, mean of surface gradient (p, q) error, and CPU timing. Each algorithm works well for certain images, but performs poorly for others. In general, minimization approaches are more robust, while the other approaches are faster.
Citations
More filters
Book•
30 Sep 2010
TL;DR: Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images and takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene.
Abstract: Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.
4,146 citations
Cites background or methods from "Shape-from-shading: a survey"
...2: Synthetic shape from shading example (Zhang et al. 1999a): (a–d) shaded images, with light from in front (0, 0, 1) and from the front right (1, 0, 1); (e–f) corresponding shape from shading reconstructions using the technique of Tsai and Shah (1994)....
[...]
...1: Some examples of shape cues and 3D modeling techniques: (a) shaded image (Zhang et al. 1999a); (b) texture gradient (Garding 1992); (c) real-time depth from focus (Nayar et al....
[...]
TL;DR: It is found that on a database of more than 100 categories, the Bayesian approach produces informative models when the number of training examples is too small for other methods to operate successfully.
Abstract: Learning visual models of object categories notoriously requires hundreds or thousands of training examples. We show that it is possible to learn much information about a category from just one, or a handful, of images. The key insight is that, rather than learning from scratch, one can take advantage of knowledge coming from previously learned categories, no matter how different these categories might be. We explore a Bayesian implementation of this idea. Object categories are represented by probabilistic models. Prior knowledge is represented as a probability density function on the parameters of these models. The posterior model for an object category is obtained by updating the prior in the light of one or more observations. We test a simple implementation of our algorithm on a database of 101 diverse object categories. We compare category models learned by an implementation of our Bayesian approach to models learned from by maximum likelihood (ML) and maximum a posteriori (MAP) methods. We find that on a database of more than 100 categories, the Bayesian approach produces informative models when the number of training examples is too small for other methods to operate successfully.
2,976 citations
TL;DR: Future directions such as the "print-it-all" paradigm, that have the potential to re-imagine current research and spawn completely new avenues for exploration are pointed out.
Abstract: Additive manufacturing (AM) is poised to bring about a revolution in the way products are designed, manufactured, and distributed to end users. This technology has gained significant academic as well as industry interest due to its ability to create complex geometries with customizable material properties. AM has also inspired the development of the maker movement by democratizing design and manufacturing. Due to the rapid proliferation of a wide variety of technologies associated with AM, there is a lack of a comprehensive set of design principles, manufacturing guidelines, and standardization of best practices. These challenges are compounded by the fact that advancements in multiple technologies (for example materials processing, topology optimization) generate a "positive feedback loop" effect in advancing AM. In order to advance research interest and investment in AM technologies, some fundamental questions and trends about the dependencies existing in these avenues need highlighting. The goal of our review paper is to organize this body of knowledge surrounding AM, and present current barriers, findings, and future trends significantly to the researchers. We also discuss fundamental attributes of AM processes, evolution of the AM industry, and the affordances enabled by the emergence of AM in a variety of areas such as geometry processing, material design, and education. We conclude our paper by pointing out future directions such as the "print-it-all" paradigm, that have the potential to re-imagine current research and spawn completely new avenues for exploration. The fundamental attributes and challenges/barriers of Additive Manufacturing (AM).The evolution of research on AM with a focus on engineering capabilities.The affordances enabled by AM such as geometry, material and tools design.The developments in industry, intellectual property, and education-related aspects.The important future trends of AM technologies.
1,792 citations
01 Oct 2016
TL;DR: A fully convolutional architecture, encompassing residual learning, to model the ambiguous mapping between monocular images and depth maps is proposed and a novel way to efficiently learn feature map up-sampling within the network is presented.
Abstract: This paper addresses the problem of estimating the depth map of a scene given a single RGB image. We propose a fully convolutional architecture, encompassing residual learning, to model the ambiguous mapping between monocular images and depth maps. In order to improve the output resolution, we present a novel way to efficiently learn feature map up-sampling within the network. For optimization, we introduce the reverse Huber loss that is particularly suited for the task at hand and driven by the value distributions commonly present in depth maps. Our model is composed of a single architecture that is trained end-to-end and does not rely on post-processing techniques, such as CRFs or other additional refinement steps. As a result, it runs in real-time on images or videos. In the evaluation, we show that the proposed model contains fewer parameters and requires fewer training data than the current state of the art, while outperforming all approaches on depth estimation. Code and models are publicly available.
1,677 citations
Cites background from "Shape-from-shading: a survey"
...ing image pairs of the same scene to reconstruct 3D shapes. In the single-view case, most approaches relied on motion (Structure-from-Motion [34]) or different shooting conditions (Shape-from-Shading [39], Shape-from-Defocus [33]). Despite the ambiguities that arise in lack of such information, but inspired by the analogy to human depth perception from monocular cues, depth map prediction from a singl...
[...]
...ls and, in turn, estimate depth via triangulation from pairs of consecutive views. Alternatively to motion, other working assumptions can be used to estimate depth, such as variations in illumination [39] or focus [33]. In absence of such environmental assumptions, depth estimation from a single image of a generic scene is an ill-posed problem, due to the inherent ambiguity of mapping an intensity or ...
[...]
TL;DR: This work considers the problem of estimating detailed 3D structure from a single still image of an unstructured environment and uses a Markov random field (MRF) to infer a set of "plane parameters" that capture both the 3D location and 3D orientation of the patch.
Abstract: We consider the problem of estimating detailed 3D structure from a single still image of an unstructured environment. Our goal is to create 3D models that are both quantitatively accurate as well as visually pleasing. For each small homogeneous patch in the image, we use a Markov random field (MRF) to infer a set of "plane parametersrdquo that capture both the 3D location and 3D orientation of the patch. The MRF, trained via supervised learning, models both image depth cues as well as the relationships between different parts of the image. Other than assuming that the environment is made up of a number of small planes, our model makes no explicit assumptions about the structure of the scene; this enables the algorithm to capture much more detailed 3D structure than does prior art and also give a much richer experience in the 3D flythroughs created using image-based rendering, even for scenes with significant nonvertical structure. Using this approach, we have created qualitatively correct 3D models for 64.9 percent of 588 images downloaded from the Internet. We have also extended our model to produce large-scale 3D models from a few images.
1,522 citations
References
More filters
01 Jan 1994
TL;DR: The Diskette v 2.06, 3.5''[1.44M] for IBM PC, PS/2 and compatibles [DOS] Reference Record created on 2004-09-07, modified on 2016-08-08.
Abstract: Note: Includes bibliographical references, 3 appendixes and 2 indexes.- Diskette v 2.06, 3.5''[1.44M] for IBM PC, PS/2 and compatibles [DOS] Reference Record created on 2004-09-07, modified on 2016-08-08
19,881 citations
TL;DR: Human visual perception and the fundamental laws of optics are considered in the development of a shading rule that provides better quality and increased realism in generated images.
Abstract: The quality of computer generated images of three-dimensional scenes depends on the shading technique used to paint the objects on the cathode-ray tube screen. The shading algorithm itself depends in part on the method for modeling the object, which also determines the hidden surface algorithm. The various methods of object modeling, shading, and hidden surface removal are thus strongly interconnected. Several shading techniques corresponding to different methods of object modeling and the related hidden surface algorithms are presented here. Human visual perception and the fundamental laws of optics are considered in the development of a shading rule that provides better quality and increased realism in generated images.
3,393 citations
"Shape-from-shading: a survey" refers methods in this paper
...Another model developed by Phong [40] represents the specular component of reflection as powers of the cosine of the angle between the perfect specular direction and the viewing direction....
[...]
Book•
03 Jan 1992
TL;DR: In this paper, the directional distribution of radiant flux reflected from roughened surfaces is analyzed on the basis of geometrical optics, and the analysis successfully predicts the off-specular maxima in the reflection distribution which are observed experimentally and which emerge as the incidence angle increases.
Abstract: The directional distribution of radiant flux reflected from roughened surfaces is analyzed on the basis of geometrical optics. The analytical model assumes that the surface consists of small, randomly disposed, mirror-like facets. Specular reflection from these facets plus a diffuse component due to multiple reflections and/or internal scattering are postulated as the basic mechanisms of the reflection process. The effects of shadowing and masking of facets by adjacent facets are included in the analysis. The angular distributions of reflected flux predicted by the analysis are in very good agreement with experiment for both metallic and nonmetallic surfaces. Moreover, the analysis successfully predicts the off-specular maxima in the reflection distribution which are observed experimentally and which emerge as the incidence angle increases. The model thus affords a rational explanation for the off-specular peak phenomenon in terms of mutual masking and shadowing of mirror-like, specularly reflecting surface facets.
1,715 citations
TL;DR: In this paper, the directional distribution of radiant flux reflected from roughened surfaces is analyzed on the basis of geometrical optics, and the analysis successfully predicts the off-specular maxima in the reflection distribution which are observed experimentally and which emerge as the incidence angle increases.
Abstract: The directional distribution of radiant flux reflected from roughened surfaces is analyzed on the basis of geometrical optics. The analytical model assumes that the surface consists of small, randomly disposed, mirror-like facets. Specular reflection from these facets plus a diffuse component due to multiple reflections and/or internal scattering are postulated as the basic mechanisms of the reflection process. The effects of shadowing and masking of facets by adjacent facets are included in the analysis. The angular distributions of reflected flux predicted by the analysis are in very good agreement with experiment for both metallic and nonmetallic surfaces. Moreover, the analysis successfully predicts the off-specular maxima in the reflection distribution which are observed experimentally and which emerge as the incidence angle increases. The model thus affords a rational explanation for the off-specular peak phenomenon in terms of mutual masking and shadowing of mirror-like, specularly reflecting surface facets.
1,673 citations
TL;DR: An approach for enforcing integrability, a particular implementation of the approach, an example of its application to extending an existing shape-from-shading algorithm, and experimental results showing the improvement that results from enforcingIntegrability are presented.
Abstract: An approach for enforcing integrability, a particular implementation of the approach, an example of its application to extending an existing shape-from-shading algorithm, and experimental results showing the improvement that results from enforcing integrability are presented. A possibly nonintegrable estimate of surface slopes is represented by a finite set of basis functions, and integrability is enforced by calculating the orthogonal projection onto a vector subspace spanning the set of integrable slopes. The integrability projection constraint was applied to extending an iterative shape-from-shading algorithm of M.J. Brooks and B.K.P. Horn (1985). Experimental results show that the extended algorithm converges faster and with less error than the original version. Good surface reconstructions were obtained with and without known boundary conditions and for fairly complicated surfaces. >
1,090 citations