This paper investigates two fundamental problems in computer vision: contour detection and image segmentation. We present state-of-the-art algorithms for both of these tasks. Our contour detector combines multiple local cues into a globalization framework based on spectral clustering. Our segmentation algorithm consists of generic machinery for transforming the output of any contour detector into a hierarchical region tree. In this manner, we reduce the problem of image segmentation to that of contour detection. Extensive experimental evaluation demonstrates that both our contour detection and segmentation methods significantly outperform competing algorithms. The automatically generated hierarchical segmentations can be interactively refined by user-specified annotations. Computation at multiple image resolutions provides a means of coupling our system to recognition applications.

/pdf/contour-detection-and-hierarchical-image-segmentation-2ejknw01zo.pdf

Contour Detection and Hierarchical Image Segmentation

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

Predicting depth is an essential component in understanding the 3D geometry of a scene. While for stereo images local correspondence suffices for estimation, finding depth relations from a single image is less straightforward, requiring integration of both global and local information from various cues. Moreover, the task is inherently ambiguous, with a large source of uncertainty coming from the overall scale. In this paper, we present a new method that addresses this task by employing two deep network stacks: one that makes a coarse global prediction based on the entire image, and another that refines this prediction locally. We also apply a scale-invariant error to help measure depth relations rather than scale. By leveraging the raw datasets as large sources of training data, our method achieves state-of-the-art results on both NYU Depth and KITTI, and matches detailed depth boundaries without the need for superpixelation.

/pdf/depth-map-prediction-from-a-single-image-using-a-multi-scale-260o56cfli.pdf

Depth Map Prediction from a Single Image using a Multi-Scale Deep Network

This paper addresses the problem of estimating the depth map of a scene given a single RGB image. We propose a fully convolutional architecture, encompassing residual learning, to model the ambiguous mapping between monocular images and depth maps. In order to improve the output resolution, we present a novel way to efficiently learn feature map up-sampling within the network. For optimization, we introduce the reverse Huber loss that is particularly suited for the task at hand and driven by the value distributions commonly present in depth maps. Our model is composed of a single architecture that is trained end-to-end and does not rely on post-processing techniques, such as CRFs or other additional refinement steps. As a result, it runs in real-time on images or videos. In the evaluation, we show that the proposed model contains fewer parameters and requires fewer training data than the current state of the art, while outperforming all approaches on depth estimation. Code and models are publicly available.

Deeper Depth Prediction with Fully Convolutional Residual Networks

We consider the problem of estimating detailed 3D structure from a single still image of an unstructured environment. Our goal is to create 3D models that are both quantitatively accurate as well as visually pleasing. For each small homogeneous patch in the image, we use a Markov random field (MRF) to infer a set of "plane parametersrdquo that capture both the 3D location and 3D orientation of the patch. The MRF, trained via supervised learning, models both image depth cues as well as the relationships between different parts of the image. Other than assuming that the environment is made up of a number of small planes, our model makes no explicit assumptions about the structure of the scene; this enables the algorithm to capture much more detailed 3D structure than does prior art and also give a much richer experience in the 3D flythroughs created using image-based rendering, even for scenes with significant nonvertical structure. Using this approach, we have created qualitatively correct 3D models for 64.9 percent of 588 images downloaded from the Internet. We have also extended our model to produce large-scale 3D models from a few images.

/pdf/make3d-learning-3d-scene-structure-from-a-single-still-image-17ontvgbdj.pdf

Make3D: Learning 3D Scene Structure from a Single Still Image

We consider the task of depth estimation from a single monocular image. We take a supervised learning approach to this problem, in which we begin by collecting a training set of monocular images (of unstructured outdoor environments which include forests, trees, buildings, etc.) and their corresponding ground-truth depthmaps. Then, we apply supervised learning to predict the depthmap as a function of the image. Depth estimation is a challenging problem, since local features alone are insufficient to estimate depth at a point, and one needs to consider the global context of the image. Our model uses a discriminatively-trained Markov Random Field (MRF) that incorporates multiscale local- and global-image features, and models both depths at individual points as well as the relation between depths at different points. We show that, even on unstructured scenes, our algorithm is frequently able to recover fairly accurate depthmaps.

/pdf/learning-depth-from-single-monocular-images-vwxjxf9l1m.pdf

Learning Depth from Single Monocular Images

We consider the task of 3-d depth estimation from a single still image. We take a supervised learning approach to this problem, in which we begin by collecting a training set of monocular images (of unstructured indoor and outdoor environments which include forests, sidewalks, trees, buildings, etc.) and their corresponding ground-truth depthmaps. Then, we apply supervised learning to predict the value of the depthmap as a function of the image. Depth estimation is a challenging problem, since local features alone are insufficient to estimate depth at a point, and one needs to consider the global context of the image. Our model uses a hierarchical, multiscale Markov Random Field (MRF) that incorporates multiscale local- and global-image features, and models the depths and the relation between depths at different points in the image. We show that, even on unstructured scenes, our algorithm is frequently able to recover fairly accurate depthmaps. We further propose a model that incorporates both monocular cues and stereo (triangulation) cues, to obtain significantly more accurate depth estimates than is possible using either monocular or stereo cues alone.

/pdf/3-d-depth-reconstruction-from-a-single-still-image-2v1nfwlktm.pdf

3-D Depth Reconstruction from a Single Still Image

Hybrid peer-to-peer data communication and management is described, including querying a plurality of elements (302-316, 318, 324- 332 and 336-344) in an environment (300) for state data, the state data describes a state of the environment (300) and an activity occurring in the environment (300), receiving the state data from one or more of the plurality of elements (302-316, 318, 324-332 and 336-344) in response to the querying, analyzing the state data to determine an environmental change, the updated state data is generate describing the environmental change, and using the updated state data to modify the environment (300) by dynamically allocating the plurality of elements to process a simulation, the plurality of elements (302-316, 318, 324-332 and 336-344) being dynamically allocated based on available processing capacity provided by the plurality of elements (302-316, 318, 324-332 and 336-344).

Hybrid peer-to-peer data communication and management

Three-dimensional image data is generated. According to an example embodiment, three-dimensional depth information is estimated from a still image. A set of monocular images and their corresponding ground-truth depth maps are used to determine a relationship between monocular image features and the depth of image points. For different points in a particular image, the determined relationship is used together with local and global image features including monocular cues to determine relative depths of the points.

Arrangement and method for three-dimensional depth image construction

A machine may be configured to access an image of an item described by a description of the item. The machine may determine an image quality score of the image based on an analysis of the image. A request for search results that pertain to the description may be received by the machine, and the machine may present a search result that references the item's image, based on its image quality score. Also, the machine may access images of items and descriptions of items and generate a set of most frequent text tokens included in the item descriptions. The machine may identify an image feature exhibited by an item's image and determine that a text token from the corresponding item description matches one of the most frequent text tokens. A data structure may be generated by the machine to correlate the identified image feature with the text token.

Sung Hwan Chung

Papers

Learning Depth from Single Monocular Images

3-D Depth Reconstruction from a Single Still Image

Hybrid peer-to-peer data communication and management

Arrangement and method for three-dimensional depth image construction

Image-based popularity prediction