Showing papers by "Andrés Bruhn published in 2005"

Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods

Abstract: Differential methods belong to the most widely used techniques for optic flow computation in image sequences. They can be classified into local methods such as the Lucas-Kanade technique or Bigun's structure tensor method, and into global methods such as the Horn/Schunck approach and its extensions. Often local methods are more robust under noise, while global techniques yield dense flow fields. The goal of this paper is to contribute to a better understanding and the design of novel differential methods in four ways: (i) We juxtapose the role of smoothing/regularisation processes that are required in local and global differential methods for optic flow computation. (ii) This discussion motivates us to describe and evaluate a novel method that combines important advantages of local and global approaches: It yields dense flow fields that are robust against noise. (iii) Spatiotemporal and nonlinear extensions as well as multiresolution frameworks are presented for this hybrid method. (iv) We propose a simple confidence measure for optic flow methods that minimise energy functionals. It allows to sparsify a dense flow field gradually, depending on the reliability required for the resulting flow. Comparisons with experiments from the literature demonstrate the favourable performance of the proposed methods and the confidence measure.

Variational optical flow computation in real time

Abstract: This paper investigates the usefulness of bidirectional multigrid methods for variational optical flow computations. Although these numerical schemes are among the fastest methods for solving equation systems, they are rarely applied in the field of computer vision. We demonstrate how to employ those numerical methods for the treatment of variational optical flow formulations and show that the efficiency of this approach even allows for real-time performance on standard PCs. As a representative for variational optic flow methods, we consider the recently introduced combined local-global method. It can be considered as a noise-robust generalization of the Horn and Schunck technique. We present a decoupled, as well as a coupled, version of the classical Gau/spl szlig/-Seidel solver, and we develop several multigrid implementations based on a discretization coarse grid approximation. In contrast, with standard bidirectional multigrid algorithms, we take advantage of intergrid transfer operators that allow for nondyadic grid hierarchies. As a consequence, no restrictions concerning the image size or the number of traversed levels have to be imposed. In the experimental section, we juxtapose the developed multigrid schemes and demonstrate their superior performance when compared to unidirectional multigrid methods and nonhierachical solvers. For the well-known 316/spl times/252 Yosemite sequence, we succeeded in computing the complete set of dense flow fields in three quarters of a second on a 3.06-GHz Pentium4 PC. This corresponds to a frame rate of 18 flow fields per second which outperforms the widely-used Gau/spl szlig/-Seidel method by almost three orders of magnitude.

Towards ultimate motion estimation: combining highest accuracy with real-time performance

Abstract: Although variational methods are among the most accurate techniques for estimating the optical flow, they have not yet entered the field of real-time vision Main reason is the great popularity of standard numerical schemes that are easy to implement, however, at the expense of being too slow for real-time performance In our paper we address this problem in two ways: (i) we present an improved version of the highly accurate technique of Brox et al (2004) Thereby we show that a separate robustification of the constancy assumptions is very useful, in particular if the I-norm is used as penalizer As a result, a method is obtained that yields the lowest angular errors in the literature, (ii) We develop an efficient numerical scheme for the proposed approach that allows real-time performance for sequences of size 160 /spl times/ 720 To this end, we combine two hierarchical strategies: a coarse-to-fine warping strategy as implementation of a fixed point iteration for a non-convex optimisation problem and a nonlinear full multigrid method - a so called full approximation scheme (FAS) - for solving the highly nonlinear equation systems at each warping level In the experimental section the advantage of the proposed approach becomes obvious: Outperforming standard numerical schemes by two orders of magnitude frame rates of six high quality flow fields per second are obtained on a 306 GHz Pentium4 PC

Optic flow goes stereo: a variational method for estimating discontinuity-preserving dense disparity maps

Abstract: We present a novel variational method for estimating dense disparity maps from stereo images. It integrates the epipolar constraint into the currently most accurate optic flow method (Brox et al. 2004). In this way, a new approach is obtained that offers several advantages compared to existing variational methods: (i) It preservers discontinuities very well due to the use of the total variation as solution-driven regulariser. (ii) It performs favourably under noise since it uses a robust function to penalise deviations from the data constraints. (iii) Its minimisation via a coarse-to-fine strategy can be theoretically justified. Experiments with both synthetic and real-world data show the excellent performance and the noise robustness of our approach.

Towards PDE-Based image compression

Abstract: While methods based on partial differential equations (PDEs) and variational techniques are powerful tools for denoising and inpainting digital images, their use for image compression was mainly focussing on pre- or postprocessing so far. In our paper we investigate their potential within the decoding step. We start with the observation that edge-enhancing diffusion (EED), an anisotropic nonlinear diffusion filter with a diffusion tensor, is well-suited for scattered data interpolation: Even when the interpolation data are very sparse, good results are obtained that respect discontinuities and satisfy a maximum–minimum principle. This property is exploited in our studies on PDE-based image compression. We use an adaptive triangulation method based on B-tree coding for removing less significant pixels from the image. The remaining points serve as scattered interpolation data for the EED process. They can be coded in a compact and elegant way that reflects the B-tree structure. Our experiments illustrate that for high compression rates and non-textured images, this PDE-based approach gives visually better results than the widely-used JPEG coding.

Discontinuity-preserving computation of variational optic flow in real-time

Abstract: Variational methods are very popular for optic flow computation: They yield dense flow fields and perform well if they are adapted such that they respect discontinuities in the image sequence or the flow field. Unfortunately, this adaptation results in high computational complexity. In our paper we show that it is possible to achieve real-time performance for these methods if bidirectional multigrid strategies are used. To this end, we study two prototypes: i) For the anisotropic image-driven technique of Nagel and Enkelmann that results in a linear system of equations we derive a regular full multigrid scheme. ii) For an isotropic flow-driven approach with total variation (TV) regularisation that requires to solve a nonlinear system of equations we develop a full multigrid strategy based on a full approximation scheme (FAS). Experiments for sequences of size 160 × 120 demonstrate the excellent performance of the proposed numerical schemes. With frame rates of 6 and 12 dense flow fields per second, respectively, both implementations outperform corresponding modified explicit schemes by two to three orders of magnitude. Thus, for the first time ever, real-time performance can be achieved for these high quality methods.

Domain decomposition for variational optical-flow computation

Abstract: We present an approach to parallel variational optical-flow computation by using an arbitrary partition of the image plane and iteratively solving related local variational problems associated with each subdomain. The approach is particularly suited for implementations on PC clusters because interprocess communication is minimized by restricting the exchange of data to a lower dimensional interface. Our mathematical formulation supports various generalizations to linear/nonlinear convex variational approaches, three-dimensional image sequences, spatiotemporal regularization, and unstructured geometries and triangulations. Results concerning the effects of interface preconditioning, as well as runtime and communication volume measurements on a PC cluster, are presented. Our approach provides a major step toward real-time two-dimensional image processing using off-the-shelf PC hardware and facilitates the efficient application of variational approaches to large-scale image processing problems.

Morphology for Higher-Dimensional Tensor Data Via Loewner Ordering

Abstract: The operators of greyscale morphology rely on the notions of maximum and minimum which regrettably are not directly available for tensor-valued data since the straightforward component-wise approach fails.

Mathematical morphology for tensor data induced by the Loewner orderingin higher dimensions

Abstract: Positive semidefinite matrix fields are becoming increasingly important in digital imaging. One reason for this tendency consists of the introduction of diffusion tensor magnetic resonance imaging (DTMRI). In order to perform shape analysis, enhancement or segmentation of such tensor fields, appropriate image processing tools must be developed. This paper extends fundamental morphological operations to the matrix-valued setting. We start by presenting novel definitions for the maximum and minimum of a set of matrices since these notions lie at the heart of the morphological operations. In contrast to naive approaches like the component-wise maximum or minimum of the matrix channels, our approach is based on the Loewner ordering for symmetric matrices. The notions of maximum and minimum deduced from this partial ordering satisfy desirable properties such as rotation invariance, preservation of positive semidefiniteness, and continuous dependence on the input data. We introduce erosion, dilation, opening, closing, top hats, morphological derivatives, shock filters, and mid-range filters for positive semidefinite matrix-valued images. These morphological operations incorporate information simultaneously from all matrix channels rather than treating them independently. Experiments on DT-MRI images with ball- and rod-shaped structuring elements illustrate the properties and performance of our morphological operators for matrix-valued data.