Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

This paper describes the semiglobal matching (SGM) stereo method. It uses a pixelwise, mutual information (Ml)-based matching cost for compensating radiometric differences of input images. Pixelwise matching is supported by a smoothness constraint that is usually expressed as a global cost function. SGM performs a fast approximation by pathwise optimizations from all directions. The discussion also addresses occlusion detection, subpixel refinement, and multibaseline matching. Additionally, postprocessing steps for removing outliers, recovering from specific problems of structured environments, and the interpolation of gaps are presented. Finally, strategies for processing almost arbitrarily large images and fusion of disparity images using orthographic projection are proposed. A comparison on standard stereo images shows that SGM is among the currently top-ranked algorithms and is best, if subpixel accuracy is considered. The complexity is linear to the number of pixels and disparity range, which results in a runtime of just 1-2 seconds on typical test images. An in depth evaluation of the Ml-based matching cost demonstrates a tolerance against a wide range of radiometric transformations. Finally, examples of reconstructions from huge aerial frame and pushbroom images demonstrate that the presented ideas are working well on practical problems.

/pdf/stereo-processing-by-semiglobal-matching-and-mutual-147l7nvp3i.pdf

Stereo Processing by Semiglobal Matching and Mutual Information

A wide range of methods for analysis of airborne- and satellite-derived imagery continues to be proposed and assessed. In this paper, we review remote sensing implementations of support vector machines (SVMs), a promising machine learning methodology. This review is timely due to the exponentially increasing number of works published in recent years. SVMs are particularly appealing in the remote sensing field due to their ability to generalize well even with limited training samples, a common limitation for remote sensing applications. However, they also suffer from parameter assignment issues that can significantly affect obtained results. A summary of empirical results is provided for various applications of over one hundred published works (as of April, 2010). It is our hope that this survey will provide guidelines for future applications of SVMs and possible areas of algorithm enhancement.

https://www.aboutgis.com/Publications/Mountrakis_SVM_review_in_remote_sensing_ISPRS2010.pdf

Support vector machines in remote sensing: A review

We propose a novel deep learning architecture for regressing disparity from a rectified pair of stereo images. We leverage knowledge of the problem’s geometry to form a cost volume using deep feature representations. We learn to incorporate contextual information using 3-D convolutions over this volume. Disparity values are regressed from the cost volume using a proposed differentiable soft argmin operation, which allows us to train our method end-to-end to sub-pixel accuracy without any additional post-processing or regularization. We evaluate our method on the Scene Flow and KITTI datasets and on KITTI we set a new stateof-the-art benchmark, while being significantly faster than competing approaches.

End-to-End Learning of Geometry and Context for Deep Stereo Regression

In this paper the main problems and the available solutions are addressed for the generation of 3D models from terrestrial images. Close range photogrammetry has dealt for many years with manual or automatic image measurements for precise 3D modelling. Nowadays 3D scanners are also becoming a standard source for input data in many application areas, but image-based modelling still remains the most complete, economical, portable, flexible and widely used approach. In this paper the full pipeline is presented for 3D modelling from terrestrial image data, considering the different approaches and analysing all the steps involved.

/pdf/image-based-3d-modelling-a-review-5msqumyd5o.pdf

Image-based 3d modelling: a review

A novel stereo matching algorithm is proposed that utilizes color segmentation on the reference image and a self-adapting matching score that maximizes the number of reliable correspondences. The scene structure is modeled by a set of planar surface patches which are estimated using a new technique that is more robust to outliers. Instead of assigning a disparity value to each pixel, a disparity plane is assigned to each segment. The optimal disparity plane labeling is approximated by applying belief propagation. Experimental results using the Middlebury stereo test bed demonstrate the superior performance of the proposed method

/pdf/segment-based-stereo-matching-using-belief-propagation-and-a-1ljeo1iwnh.pdf

Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure

This paper describes the fusion of information extracted from multispectral digital aerial images for highly automatic 3D map generation. The proposed approach integrates spectral classification and 3D reconstruction techniques. The multispectral digital aerial images consist of a high resolution panchromatic channel as well as lower resolution RGB and near infrared (NIR) channels and form the basis for information extraction. Our land use classification is a 2-step approach that uses RGB and NIR images for an initial classification and the panchromatic images as well as a digital surface model (DSM) for a refined classification. The DSM is generated from the high resolution panchromatic images of a specific photo mission. Based on the aerial triangulation using area and feature-based points of interest the algorithms are able to generate a dense DSM by a dense image matching procedure. Afterwards a true ortho image for classification, panchromatic or color input images can be computed. In a last step specific layers for buildings and vegetation are generated and the classification is updated.

Towards 3D map generation from digital aerial images

We present a simple and universal camera calibration method. Instead of extensive setups we are exploiting the accurate angular positions of fixed stars. High precision is achieved by compensating the interfering error sources. Our approach uses a star catalog and requires a single input image only. No additional user input information such as focal length, exposure date or position is required. Fully automatic processing and fast convergence is achieved by performing three consecutive steps. First, a star segmentation and centroid finding algorithm extracts the sub-pixel positions of the luminaries. Second, an initial solution for the most essential parameters is determined by combinatorial analysis. Finally, the Levenberg-Marquardt algorithm is applied to solve the resulting non-linear system. Experimental results with several digital consumer cameras demonstrate high robustness and accuracy. The introduced method is advisable for applications where large calibration targets are required.

/pdf/camera-calibration-from-a-single-night-sky-image-3kwn4v55nq.pdf

Camera calibration from a single night sky image

Vertex programs and pixel shaders found in modern graphics hardware are commonly used to enhance the realism of rendered scenes. Recently these hardware facilities were exploited to obtain interactive non-photorealistic effects and to perform low-level image processing tasks like texture filtering for volume visualization. We exploit modern graphics hardware to accomplish the higher level vision task of dense stereo reconstruction. In our system almost every stage of the matching procedure is executed on 3D graphics hardware, therefore utilizing the parallel vertex and pixel pipelines. Our implementation performs accurate calculations and does not suffer from the limited precision of color channels. On state of the art PC hardware our algorithm requires less than one second to reconstruct a dense mesh with subpixel accuracy for input images with one megapixel resolution.

/pdf/accurate-dense-stereo-reconstruction-using-graphics-hardware-4qmrrsy068.pdf

Andreas Klaus

Papers

Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure

Towards 3D map generation from digital aerial images

Camera calibration from a single night sky image

Accurate Dense Stereo Reconstruction using Graphics Hardware

Segmentation of Building Models From Dense 3D Point-Clouds