Showing papers by "Thomas Brox published in 2011"
TL;DR: A way to approach the problem of dense optical flow estimation by integrating rich descriptors into the variational optical flow setting, while reaching out to new domains of motion analysis where the requirement of dense sampling in time is no longer satisfied is presented.
Abstract: Optical flow estimation is classically marked by the requirement of dense sampling in time. While coarse-to-fine warping schemes have somehow relaxed this constraint, there is an inherent dependency between the scale of structures and the velocity that can be estimated. This particularly renders the estimation of detailed human motion problematic, as small body parts can move very fast. In this paper, we present a way to approach this problem by integrating rich descriptors into the variational optical flow setting. This way we can estimate a dense optical flow field with almost the same high accuracy as known from variational optical flow, while reaching out to new domains of motion analysis where the requirement of dense sampling in time is no longer satisfied.
1,429 citations
06 Nov 2011
TL;DR: A variational method to obtain dense segmentations from sparse trajectory clusters by propagating information with a hierarchical, nonlinear diffusion process that runs in the continuous domain but takes superpixels into account.
Abstract: Point trajectories have emerged as a powerful means to obtain high quality and fully unsupervised segmentation of objects in video shots. They can exploit the long term motion difference between objects, but they tend to be sparse due to computational reasons and the difficulty in estimating motion in homogeneous areas. In this paper we introduce a variational method to obtain dense segmentations from such sparse trajectory clusters. Information is propagated with a hierarchical, nonlinear diffusion process that runs in the continuous domain but takes superpixels into account. We show that this process raises the density from 3% to 100% and even increases the average precision of labels.
264 citations
TL;DR: A variational framework for the estimation of stereoscopic scene flow, i.e., the motion of points in the three-dimensional world from stereo image sequences, which partially decouple the depth estimation from the motion estimation, which has many practical advantages.
Abstract: Building upon recent developments in optical flow and stereo matching estimation, we propose a variational framework for the estimation of stereoscopic scene flow, i.e., the motion of points in the three-dimensional world from stereo image sequences. The proposed algorithm takes into account image pairs from two consecutive times and computes both depth and a 3D motion vector associated with each point in the image. In contrast to previous works, we partially decouple the depth estimation from the motion estimation, which has many practical advantages. The variational formulation is quite flexible and can handle both sparse or dense disparity maps. The proposed method is very efficient; with the depth map being computed on an FPGA, and the scene flow computed on the GPU, the proposed algorithm runs at frame rates of 20 frames per second on QVGA images (320×240 pixels). Furthermore, we present solutions to two important problems in scene flow estimation: violations of intensity consistency between input images, and the uncertainty measures for the scene flow result.
206 citations
20 Jun 2011
TL;DR: This work proposes a contour and region detector for video data that exploits motion cues and distinguishes occlusion boundaries from internal boundaries based on optical flow, and outperforms the state-of-the-art on the benchmark of Stein and Hebert.
Abstract: In this work, we propose a contour and region detector for video data that exploits motion cues and distinguishes occlusion boundaries from internal boundaries based on optical flow. This detector outperforms the state-of-the-art on the benchmark of Stein and Hebert [24], improving average precision from. 58 to. 72. Moreover, the optical flow on and near occlusion boundaries allows us to assign a depth ordering to the adjacent regions. To evaluate performance on this edge-based figure/ground labeling task, we introduce a new video dataset that we believe will support further research in the field by allowing quantitative comparison of computational models for occlusion boundary detection, depth ordering and segmentation in video sequences.
171 citations
20 Jun 2011
TL;DR: This paper builds upon the part-based pose-let detector, which can predict masks for numerous parts of an object, and extends poselets to 19 other categories apart from person.
Abstract: In this paper, we propose techniques to make use of two complementary bottom-up features, image edges and texture patches, to guide top-down object segmentation towards higher precision. We build upon the part-based pose-let detector, which can predict masks for numerous parts of an object. For this purpose we extend poselets to 19 other categories apart from person. We non-rigidly align these part detections to potential object contours in the image, both to increase the precision of the predicted object mask and to sort out false positives. We spatially aggregate object information via a variational smoothing technique while ensuring that object regions do not overlap. Finally, we propose to refine the segmentation based on self-similarity defined on small image patches. We obtain competitive results on the challenging Pascal VOC benchmark. On four classes we achieve the best numbers to-date.
127 citations
09 Jun 2011
TL;DR: The presented HMRF outperforms classical MRFs as well as local classification approaches wrt.
Abstract: We present a hierarchical Markov Random Field (HMRF) for multi-label image segmentation. With such a hierarchical model, we can incorporate global knowledge into our segmentation algorithm. Solving the MRF is formulated as a MAX-SUM problem for which there exist efficient solvers based on linear programming. We show that our method allows for automatic segmentation of mast cells and their cell organelles from 2D electron microscopic recordings. The presented HMRF outperforms classical MRFs as well as local classification approaches wrt. pixelwise segmentation accuracy. Additionally, the resulting segmentations are much more consistent regarding the region compactness.
18 citations
01 Jan 2011
TL;DR: In this paper, the authors provide additional results on multiview deblurring and accompany the work in [11] and provide images of all data sets before and after reconstruction, and give more insights into the point spread function (PSF) of the Single Plane Illumination Microscope (SPIM) theoretically and empirically.
Abstract: This report provides additional results on multiview deblurring and accompanies the work in [11]. In particular, we provide images of all data sets before and after reconstruction. Moreover, we give more insights into the point spread function (PSF) of the Single Plane Illumination Microscope (SPIM) theoretically and empirically. Finally, the influence of the number of views on the resolution in frequency space is discussed.
01 Jan 2011
TL;DR: This report provides additional results on multiview deblurring and gives more insights into the point spread function (PSF) of the Single Plane Illumination Microscope (SPIM) theoretically and empirically.
Abstract: This report provides additional results on multiview deblurring and accompanies the work in [11]. In particular, we provide images of all data sets before and after reconstruction. Moreover, we give more insights into the point spread function (PSF) of the Single Plane Illumination Microscope (SPIM) theoretically and empirically. Finally, the influence of the number of views on the resolution in frequency space is discussed.