scispace - formally typeset

Journal ArticleDOI

Novel approach to nonlinear/non-Gaussian Bayesian state estimation

01 Apr 1993-Vol. 140, Iss: 2, pp 107-113

TL;DR: An algorithm, the bootstrap filter, is proposed for implementing recursive Bayesian filters, represented as a set of random samples, which are updated and propagated by the algorithm.
Abstract: An algorithm, the bootstrap filter, is proposed for implementing recursive Bayesian filters. The required density of the state vector is represented as a set of random samples, which are updated and propagated by the algorithm. The method is not restricted by assumptions of linear- ity or Gaussian noise: it may be applied to any state transition or measurement model. A simula- tion example of the bearings only tracking problem is presented. This simulation includes schemes for improving the efficiency of the basic algorithm. For this example, the performance of the bootstrap filter is greatly superior to the standard extended Kalman filter.
Topics: Extended Kalman filter (64%), Invariant extended Kalman filter (63%), Kalman filter (62%), Filter design (61%), Gaussian noise (59%)
Citations
More filters

Journal ArticleDOI
TL;DR: Both optimal and suboptimal Bayesian algorithms for nonlinear/non-Gaussian tracking problems, with a focus on particle filters are reviewed.
Abstract: Increasingly, for many application areas, it is becoming important to include elements of nonlinearity and non-Gaussianity in order to model accurately the underlying dynamics of a physical system. Moreover, it is typically crucial to process data on-line as it arrives, both from the point of view of storage costs as well as for rapid adaptation to changing signal characteristics. In this paper, we review both optimal and suboptimal Bayesian algorithms for nonlinear/non-Gaussian tracking problems, with a focus on particle filters. Particle filters are sequential Monte Carlo methods based on point mass (or "particle") representations of probability densities, which can be applied to any state-space model and which generalize the traditional Kalman filtering methods. Several variants of the particle filter such as SIR, ASIR, and RPF are introduced within a generic framework of the sequential importance sampling (SIS) algorithm. These are discussed and compared with the standard EKF through an illustrative example.

10,977 citations


Cites methods or result from "Novel approach to nonlinear/non-Gau..."

  • ...[14]. This sequential MC (SMC) approach is known variously as bootstrap filtering [ 17 ], the condensation algorithm [29], particle filtering [6], interacting particle approximations [10], [11], and survival of the fittest [24]....

    [...]

  • ...1) Sampling Importance Resampling Filter: The SIR filter proposed in [ 17 ] is an MC method that can be applied to recursive Bayesian filtering problems....

    [...]

  • ...We use and . This example has been analyzed before in many publications [5], [ 17 ], [25]....

    [...]


Christopher M. Bishop1Institutions (1)
01 Jan 2006
Abstract: Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

10,141 citations


Book
24 Aug 2012
TL;DR: This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach, and is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.
Abstract: Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

7,045 citations


Cites background from "Novel approach to nonlinear/non-Gau..."

  • ...(The original bootstrap filter (Gordon 1993) resampled at every step, but this is suboptimal....

    [...]


BookDOI
01 Jan 2001
TL;DR: This book presents the first comprehensive treatment of Monte Carlo techniques, including convergence results and applications to tracking, guidance, automated target recognition, aircraft navigation, robot navigation, econometrics, financial modeling, neural networks, optimal control, optimal filtering, communications, reinforcement learning, signal enhancement, model averaging and selection.
Abstract: Monte Carlo methods are revolutionizing the on-line analysis of data in fields as diverse as financial modeling, target tracking and computer vision. These methods, appearing under the names of bootstrap filters, condensation, optimal Monte Carlo filters, particle filters and survival of the fittest, have made it possible to solve numerically many complex, non-standard problems that were previously intractable. This book presents the first comprehensive treatment of these techniques, including convergence results and applications to tracking, guidance, automated target recognition, aircraft navigation, robot navigation, econometrics, financial modeling, neural networks, optimal control, optimal filtering, communications, reinforcement learning, signal enhancement, model averaging and selection, computer vision, semiconductor design, population biology, dynamic Bayesian networks, and time series analysis. This will be of great value to students, researchers and practitioners, who have some basic knowledge of probability. Arnaud Doucet received the Ph. D. degree from the University of Paris-XI Orsay in 1997. From 1998 to 2000, he conducted research at the Signal Processing Group of Cambridge University, UK. He is currently an assistant professor at the Department of Electrical Engineering of Melbourne University, Australia. His research interests include Bayesian statistics, dynamic models and Monte Carlo methods. Nando de Freitas obtained a Ph.D. degree in information engineering from Cambridge University in 1999. He is presently a research associate with the artificial intelligence group of the University of California at Berkeley. His main research interests are in Bayesian statistics and the application of on-line and batch Monte Carlo methods to machine learning. Neil Gordon obtained a Ph.D. in Statistics from Imperial College, University of London in 1993. He is with the Pattern and Information Processing group at the Defence Evaluation and Research Agency in the United Kingdom. His research interests are in time series, statistical data analysis, and pattern recognition with a particular emphasis on target tracking and missile guidance.

6,458 citations


Cites background or methods from "Novel approach to nonlinear/non-Gau..."

  • ...This includes the multinomial sampling originally proposed in (Gordon et al. 1993), residual resampling (Higuchi 1997, Liu and Chen 1998) and minimum variance sampling (Carpenter, Clifford and Fearnhead 1999a, Crisan and Lyons 1999, Kitagawa 1996)....

    [...]

  • ...In (Del Moral and Guionnet 1998b), the algorithm under study is the standard bootstrap filter (Gordon et al. 1993), which is based on an exploration of the space according to the natural dynamics of the system...

    [...]

  • ...In particular, we show that the auxiliary particle filtering methods of (Pitt and Shephard: this volume) fall into the same general class of algorithms as the standard bootstrap filter of (Gordon et al. 1993)....

    [...]

  • ...The introduction of this key step in (Gordon et al. 1993) led to the first operational SMC method....

    [...]

  • ...It is possible to carry out sample regeneration using a mixture approximation (Gordon et al. 1993), see also Liu and West (2001: this volume)....

    [...]


Journal ArticleDOI
Michael Isard1, Andrew Blake1Institutions (1)
TL;DR: The Condensation algorithm uses “factored sampling”, previously applied to the interpretation of static images, in which the probability distribution of possible interpretations is represented by a randomly generated set.
Abstract: The problem of tracking curves in dense visual clutter is challenging. Kalman filtering is inadequate because it is based on Gaussian densities which, being unimo dal, cannot represent simultaneous alternative hypotheses. The Condensation algorithm uses “factored sampling”, previously applied to the interpretation of static images, in which the probability distribution of possible interpretations is represented by a randomly generated set. Condensation uses learned dynamical models, together with visual observations, to propagate the random set over time. The result is highly robust tracking of agile motion. Notwithstanding the use of stochastic methods, the algorithm runs in near real-time.

5,749 citations


Cites background from "Novel approach to nonlinear/non-Gau..."

  • ...…modelling (Blake et al., 1993, 1995), object dynamics are modelled as a second order process, conveniently represented in discrete timet as a second order linear difference equation: xt − x̄ = A(xt−1 − x̄)+ Bwt (10) wherewt are independent vectors of independent standard normal variables, the…...

    [...]


References
More filters

BookDOI
01 Jan 1986
TL;DR: The Kernel Method for Multivariate Data: Three Important Methods and Density Estimation in Action.
Abstract: Introduction. Survey of Existing Methods. The Kernel Method for Univariate Data. The Kernel Method for Multivariate Data. Three Important Methods. Density Estimation in Action.

15,308 citations


Book
14 Mar 1970
Abstract: This book presents a unified treatment of linear and nonlinear filtering theory for engineers, with sufficient emphasis on applications to enable the reader to use the theory. The need for this book is twofold. First, although linear estimation theory is relatively well known, it is largely scattered in the journal literature and has not been collected in a single source. Second, available literature on the continuous nonlinear theory is quite esoteric and controversial, and thus inaccessible to engineers uninitiated in measure theory and stochastic differential equations. Furthermore, it is not clear from the available literature whether the nonlinear theory can be applied to practical engineering problems. In attempting to fill the stated needs, the author has retained as much mathematical rigor as he felt was consistent with the prime objective" to explain the theory to engineers. Thus, the author has avoided measure theory in this book by using mean square convergence, on the premise that everyone knows how to average. As a result, the author only requires of the reader background in advanced calculus, theory of ordinary differential equations, and matrix analysis.

6,533 citations


Journal ArticleDOI
Jim Q. Smith1Institutions (1)

1,213 citations


Journal ArticleDOI
D. Alspach1, H. Sorenson2Institutions (2)
TL;DR: In this paper an approximation that permits the explicit calculation of the a posteriori density from the Bayesian recursion relations is discussed and applied to the solution of the nonlinear filtering problem.
Abstract: Knowledge of the probability density function of the state conditioned on all available measurement data provides the most complete possible description of the state, and from this density any of the common types of estimates (e.g., minimum variance or maximum a posteriori) can be determined. Except in the linear Gaussian case, it is extremely difficult to determine this density function. In this paper an approximation that permits the explicit calculation of the a posteriori density from the Bayesian recursion relations is discussed and applied to the solution of the nonlinear filtering problem. In particular, it is noted that a weighted sum of Gaussian probability density functions can be used to approximate arbitrarily closely another density function. This representation provides the basis for procedure that is developed and discussed.

1,181 citations


Journal Article
Abstract: Even to the initiated, statistical calculations based on Bayes's Theorem can be daunting because of the numerical integrations required in all but the simplest applications. Moreover, from a teaching perspective, introductions to Bayesian statistics—if they are given at all—are circumscribed by these apparent calculational difficulties. Here we offer a straightforward sampling-resampling perspective on Bayesian inference, which has both pedagogic appeal and suggests easily implemented calculation strategies.

846 citations


Network Information
Performance
Metrics
No. of citations received by the Paper in previous years
YearCitations
202211
2021280
2020331
2019402
2018397
2017436