Mike Klaas

Bio: Mike Klaas is an academic researcher from University of British Columbia. The author has contributed to research in topics: Particle filter & Markov chain Monte Carlo. The author has an hindex of 5, co-authored 7 publications receiving 350 citations.

Fast particle smoothing: if I had a million particles

Abstract: We propose efficient particle smoothing methods for generalized state-spaces models. Particle smoothing is an expensive O(N2) algorithm, where N is the number of particles. We overcome this problem by integrating dual tree recursions and fast multipole techniques with forward-backward smoothers, a new generalized two-filter smoother and a maximum a posteriori (MAP) smoother. Our experiments show that these improvements can substantially increase the practicality of particle smoothing.

Toward practical N 2 Monte Carlo: the marginal particle filter

Abstract: Sequential Monte Carlo techniques are useful for state estimation in non-linear, non-Gaussian dynamic models. These methods allow us to approximate the joint posterior distribution using sequential importance sampling. In this framework, the dimension of the target distribution grows with each time step, thus it is necessary to introduce some resampling steps to ensure that the estimates provided by the algorithm have a reasonable variance. In many applications, we are only interested in the marginal filtering distribution which is defined on a space of fixed dimension. We present a Sequential Monte Carlo algorithm called the Marginal Particle Filter which operates directly on the marginal distribution, hence avoiding having to perform importance sampling on a space of growing dimension. Using this idea, we also derive an improved version of the auxiliary particle filter. We show theoretic and empirical results which demonstrate a reduction in variance over conventional particle filtering, and present techniques for reducing the cost of the marginal particle filter with N particles from O(N2) to O(N log N).

Toward Practical N2 Monte Carlo: the Marginal Particle Filter

Abstract: Sequential Monte Carlo techniques are useful for state estimation in non-linear, non-Gaussian dynamic models. These methods allow us to approximate the joint posterior distribution using sequential importance sampling. In this framework, the dimension of the target distribution grows with each time step, thus it is necessary to introduce some resampling steps to ensure that the estimates provided by the algorithm have a reasonable variance. In many applications, we are only interested in the marginal filtering distribution which is defined on a space of fixed dimension. We present a Sequential Monte Carlo algorithm called the Marginal Particle Filter which operates directly on the marginal distribution, hence avoiding having to perform importance sampling on a space of growing dimension. Using this idea, we also derive an improved version of the auxiliary particle filter. We show theoretic and empirical results which demonstrate a reduction in variance over conventional particle filtering, and present techniques for reducing the cost of the marginal particle filter with N particles from O(N2) to O(N logN).

Fast maximum a posteriori inference in Monte Carlo state spaces

Abstract: Many important algorithms for statistical inference can be expressed as a weighted maxkernel search problem. This is the case with the Viterbi algorithm for HMMs, message construction in maximum a posteriori BP (max-BP), as well as certain particlesmoothing algorithms. Previous work has focused on reducing the cost of this procedure in discrete regular grids [4]. MonteCarlo state spaces, which are vital for highdimensional inference, cannot be handled by these techniques. We present a novel dualtree based algorithm that is appliable to a wide range of kernels and shows substantial performance gains over naive computation.

Toward indicative discussion fora summarization

Abstract: Summarization of electronic discussion fora is a unique challenge; techniques that work startlingly well on monolithic documents tend to fare poorly in this informal setting. Additionally, conventional techniques ignore much of the structures that have the potential to serve as valuable features in the summarization task. We present several novel examples of such features, including the catalyst score, which is effective at identifying salient messages without looking at their content. We also describe and evaluate NewsSum, a prototype summarization system that is able to efficiently generate variable-length summarizations of Usenet threads.

A Tutorial on Particle Filtering and Smoothing: Fifteen years later

Abstract: Optimal estimation problems for non-linear non-Gaussian state-space models do not typically admit analytic solutions. Since their introduction in 1993, particle filtering methods have become a very popular class of algorithms to solve these estimation problems numerically in an online manner, i.e. recursively as observations become available, and are now routinely used in fields as diverse as computer vision, econometrics, robotics and navigation. The objective of this tutorial is to provide a complete, up-to-date survey of this field as of 2008. Basic and advanced particle methods for filtering as well as smoothing are presented.

An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo

Abstract: It is now over a decade since the pioneering contribution of Gordon (1993), which is commonly regarded as the first instance of modern sequential Monte Carlo (SMC) approaches. Initially focussed on applications to tracking and vision, these techniques are now very widespread and have had a significant impact in virtually all areas of signal and image processing concerned with Bayesian dynamical models. This paper is intended to serve both as an introduction to SMC algorithms for nonspecialists and as a reference to recent contributions in domains where the techniques are still under significant development, including smoothing, estimation of fixed parameters and use of SMC methods beyond the standard filtering contexts.

Distance Transforms of Sampled Functions

Abstract: We describe linear-time algorithms for solving a class of problems that involve transforming a cost function on a grid using spatial information. These problems can be viewed as a generalization of classical distance transforms of binary images, where the binary image is replaced by an arbitrary function on a grid. Alternatively they can be viewed in terms of the minimum convolution of two functions, which is an important operation in grayscale morphology. A consequence of our techniques is a simple and fast method for computing the Euclidean distance transform of a binary image. Our algorithms are also applicable to Viterbi decoding, belief propagation, and optimal control.

Particle Filtering in Geophysical Systems

Abstract: The application of particle filters in geophysical systems is reviewed. Some background on Bayesian filtering is provided, and the existing methods are discussed. The emphasis is on the methodology, and not so much on the applications themselves. It is shown that direct application of the basic particle filter (i.e., importance sampling using the prior as the importance density) does not work in high-dimensional systems, but several variants are shown to have potential. Approximations to the full problem that try to keep some aspects of the particle filter beyond the Gaussian approximation are also presented and discussed.

Particle filter theory and practice with positioning applications

Abstract: The particle filter (PF) was introduced in 1993 as a numerical approximation to the nonlinear Bayesian filtering problem, and there is today a rather mature theory as well as a number of successful applications described in literature. This tutorial serves two purposes: to survey the part of the theory that is most important for applications and to survey a number of illustrative positioning applications from which conclusions relevant for the theory can be drawn. The theory part first surveys the nonlinear filtering problem and then describes the general PF algorithm in relation to classical solutions based on the extended Kalman filter (EKF) and the point mass filter (PMF). Tuning options, design alternatives, and user guidelines are described, and potential computational bottlenecks are identified and remedies suggested. Finally, the marginalized (or Rao-Blackwellized) PF is overviewed as a general framework for applying the PF to complex systems. The application part is more or less a stand-alone tutorial without equations that does not require any background knowledge in statistics or nonlinear filtering. It describes a number of related positioning applications where geographical information systems provide a nonlinear measurement and where it should be obvious that classical approaches based on Kalman filters (KFs) would have poor performance. All applications are based on real data and several of them come from real-time implementations. This part also provides complete code examples.