Inspired by the recent success of methods that employ shape priors to achieve robust 3D reconstructions, we propose a novel recurrent neural network architecture that we call the 3D Recurrent Reconstruction Neural Network (3D-R2N2). The network learns a mapping from images of objects to their underlying 3D shapes from a large collection of synthetic data [13]. Our network takes in one or more images of an object instance from arbitrary viewpoints and outputs a reconstruction of the object in the form of a 3D occupancy grid. Unlike most of the previous works, our network does not require any image annotations or object class labels for training or testing. Our extensive experimental analysis shows that our reconstruction framework (i) outperforms the state-of-the-art methods for single view reconstruction, and (ii) enables the 3D reconstruction of objects in situations when traditional SFM/SLAM methods fail (because of lack of texture and/or wide baseline).

/pdf/3d-r2n2-a-unified-approach-for-single-and-multi-view-3d-55u1k3yjix.pdf

3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Extraction of three-dimensional structure of a scene from stereo images is a problem that has been studied by the computer vision community for decades. Early work focused on the fundamentals of image correspondence and stereo geometry. Stereo research has matured significantly throughout the years and many advances in computational stereo continue to be made, allowing stereo to be applied to new and more demanding problems. We review recent advances in computational stereo, focusing primarily on three important topics: correspondence methods, methods for occlusion, and real-time implementations. Throughout, we present tables that summarize and draw distinctions among key ideas and approaches. Where available, we provide comparative analyses and we make suggestions for analyses yet to be done.

https://www.cs.jhu.edu/~burschka/publications/StereoPAMI.pdf

Advances in computational stereo

Stereo correspondence methods rely on matching costs for computing the similarity of image locations. In this paper we evaluate the insensitivity of different matching costs with respect to radiometric variations of the input images. We consider both pixel-based and window-based variants and measure their performance in the presence of global intensity changes (e.g., due to gain and exposure differences), local intensity changes (e.g., due to vignetting, non-Lambertian surfaces, and varying lighting), and noise. Using existing stereo datasets with ground-truth disparities as well as six new datasets taken under controlled changes of exposure and lighting, we evaluate the different costs with a local, a semi-global, and a global stereo method.

/pdf/evaluation-of-cost-functions-for-stereo-matching-2dqk9akhtz.pdf

Evaluation of Cost Functions for Stereo Matching

Within the last several years, the field of terahertz science and technology has changed dramatically. Many new advances in the technology for generation, manipulation, and detection of terahertz radiation have revolutionized the field. Much of this interest has been inspired by the promise of valuable new applications for terahertz imaging and sensing. Among a long list of proposed uses, one finds compelling needs such as security screening and quality control, as well as whimsical notions such as counting the almonds in a bar of chocolate. This list has grown in parallel with the development of new technologies and new paradigms for imaging and sensing. Many of these proposed applications exploit the unique capabilities of terahertz radiation to penetrate common packaging materials and provide spectroscopic information about the materials within. Several of the techniques used for terahertz imaging have been borrowed from other, more well established fields such as x-ray computed tomography and synthetic aperture radar. Others have been developed exclusively for the terahertz field, and have no analogies in other portions of the spectrum. This review provides a comprehensive description of the various techniques which have been employed for terahertz image formation, as well as discussing numerous examples which illustrate the many exciting potential uses for these emerging technologies.

https://iopscience.iop.org/article/10.1088/0034-4885/70/8/R02/pdf

Imaging with terahertz radiation

WITH the ever-widening scope of modern mathematical analysis and its many ramifications, it is quite impossible to include, in a single volume of reasonable size, an adequate and exhaustive discussion of the calculus in its more advanced stages. It therefore becomes necessary, in planning a thoroughly sound course in the subject, to consider several important aspects of the vast field confronting a modern writer. The limitation of space renders the selection of subject-matter fundamentally dependent upon the aim of the course, which may or may not be related to the content of specific examination syllabuses. Logical development, too, may lead to the inclusion of many topics which, at present, may only be of academic interest, while others, of greater practical value, may have to be omitted. The experience and training of the writer may also have, more or less, a bearing on both these considerations.Advanced CalculusBy Dr. C. A. Stewart. Pp. xviii + 523. (London: Methuen and Co., Ltd., 1940.) 25s.

Advanced Calculus I

We present ordinal measures of association for image correspondence in the context of stereo. Linear correspondence measures like correlation and the sum of squared difference between intensity distributions are known to be fragile. Ordinal measures which are based on relative ordering of intensity values in windows-rank permutations-have demonstrable robustness. By using distance metrics between two rank permutations, ordinal measures are defined. These measures are independent of absolute intensity scale and invariant to monotone transformations of intensity values like gamma variation between images. We have developed simple algorithms for their efficient implementation. Experiments suggest the superiority of ordinal measures over existing techniques under nonideal conditions. These measures serve as a general tool for image matching that are applicable to other vision problems such as motion estimation and texture-based image retrieval.

/pdf/ordinal-measures-for-image-correspondence-4wu100hdws.pdf

Ordinal measures for image correspondence

We present ordinal measures for establishing image correspondence. Linear correspondence measures like correlation and the sum of squared differences are known to be fragile. Ordinal measures, which are based on relative ordering of intensity values in windows, have demonstrable robustness to depth discontinuities, occlusion and noise. The relative ordering of intensity values in each window is represented by a rank permutation which is obtained by sorting the corresponding intensity data. By using a novel distance metric between the rank permutations, we arrive at ordinal correlation coefficients. These coefficients are independent of absolute intensity scale, i.e. they are normalized measures. Further, since rank permutations are invariant to monotone transformations of the intensity values, the coefficients are unaffected by nonlinear effects like gamma variation between images. We have developed a simple algorithm for their efficient implementation. Experiments suggest the superiority of ordinal measures over existing techniques under non-ideal conditions. Though we present ordinal measures in the context of stereo, they serve as a general tool for image matching that is applicable to other vision problems such as motion estimation and image registration.

/pdf/ordinal-measures-for-visual-correspondence-1f02ql0p1y.pdf

Ordinal measures for visual correspondence

This invention provides technology (including apparatus, method, software, and/or data structures) to broadcast data from one or more content providers via one or more broadcast channels to one or more content recipients. This technology meets the different needs of the three major stakeholders—the content providers, the broadcasters, and the content recipients. The technology provides support for identifying the data to be broadcast, setting the broadcast schedule, getting the data inserted into the broadcast stream, extracting it from the broadcast stream at the receiver, and many ancillary activities such as compression, error correction, bandwidth management, bandwidth usage reporting, and content filtering at the receiver.

Three part architecture for digital television data broadcasting

A method of providing localized data includes receiving, at least one of a plurality of local broadcast sites, a broadcast stream carrying global data; and inserting, at least one of local broadcast sites, localized data associated with the one of local broadcast sites into the received broadcast stream, so as to generate a modified broadcast stream carrying the global data and the localized data at the local broadcast site. A method of providing targeted data includes receiving a broadcast including a plurality of content streams, each of the content streams carrying targeted content data having certain targeting attributes; and selecting, by a selection device, one of the content streams based on the targeting attributes and attributes of a receiving party serviced by the selection device.

Localization and targeting of data in broadcast streams

The problem of accurate depth estimation using stereo in the presence of specular reflection is addressed. Specular reflection, a fundamental and ubiquitous reflection mechanism, is viewpoint dependent and can cause large intensity differences at corresponding points, resulting in significant depth errors. We analyze the physics of specular reflection and the geometry of stereopsis which lead to a relationship between stereo vergence, surface roughness, and the likelihood of a correct match. Given a lower bound on surface roughness, an optimal binocular stereo configuration can be determined which maximizes precision in depth estimation despite specular reflection. However, surface roughness is difficult to estimate in unstructured environments. Therefore, trinocular configurations, independent of surface roughness are determined such that at each scene point visible to all sensors, at least one stereo pair can produce correct depth. We have developed a simple algorithm to reconstruct depth from the multiple stereo pairs.

Dinkar N. Bhat

Papers

Ordinal measures for image correspondence

Ordinal measures for visual correspondence

Three part architecture for digital television data broadcasting

Localization and targeting of data in broadcast streams

Stereo and Specular Reflection