Algorithms that must deal with complicated global functions of many variables often exploit the manner in which the given functions factor as a product of "local" functions, each of which depends on a subset of the variables. Such a factorization can be visualized with a bipartite graph that we call a factor graph, In this tutorial paper, we present a generic message-passing algorithm, the sum-product algorithm, that operates in a factor graph. Following a single, simple computational rule, the sum-product algorithm computes-either exactly or approximately-various marginal functions derived from the global function. A wide variety of algorithms developed in artificial intelligence, signal processing, and digital communications can be derived as specific instances of the sum-product algorithm, including the forward/backward algorithm, the Viterbi algorithm, the iterative "turbo" decoding algorithm, Pearl's (1988) belief propagation algorithm for Bayesian networks, the Kalman filter, and certain fast Fourier transform (FFT) algorithms.

/pdf/factor-graphs-and-the-sum-product-algorithm-4ow9910cb0.pdf

Factor graphs and the sum-product algorithm

The problem of tracking curves in dense visual clutter is challenging. Kalman filtering is inadequate because it is based on Gaussian densities which, being unimo dal, cannot represent simultaneous alternative hypotheses. The Condensation algorithm uses “factored sampling”, previously applied to the interpretation of static images, in which the probability distribution of possible interpretations is represented by a randomly generated set. Condensation uses learned dynamical models, together with visual observations, to propagate the random set over time. The result is highly robust tracking of agile motion. Notwithstanding the use of stochastic methods, the algorithm runs in near real-time.

/pdf/c-ondensation-conditional-density-propagation-forvisual-2zlkn4lnac.pdf

C ONDENSATION —Conditional Density Propagation forVisual Tracking

In this article, we present an overview of methods for sequential simulation from posterior distributions. These methods are of particular interest in Bayesian filtering for discrete time dynamic models that are typically nonlinear and non-Gaussian. A general importance sampling framework is developed that unifies many of the methods which have been proposed over the last few decades in several different scientific disciplines. Novel extensions to the existing methods are also proposed. We show in particular how to incorporate local linearisation methods similar to those which have previously been employed in the deterministic filtering literatures these lead to very effective importance distributions. Furthermore we describe a method which uses Rao-Blackwellisation in order to take advantage of the analytic structure present in some important classes of state-space models. In a final section we develop algorithms for prediction, smoothing and evaluation of the likelihood in dynamic models.

/pdf/on-sequential-monte-carlo-sampling-methods-for-bayesian-3bit26usft.pdf

On sequential Monte Carlo sampling methods for Bayesian filtering

Dynamic Bayesian Networks: Representation, Inference and Learning by Kevin Patrick Murphy Doctor of Philosophy in Computer Science University of California, Berkeley Professor Stuart Russell, Chair Modelling sequential data is important in many areas of science and engineering. Hidden Markov models (HMMs) and Kalman filter models (KFMs) are popular for this because they are simple and flexible. For example, HMMs have been used for speech recognition and bio-sequence analysis, and KFMs have been used for problems ranging from tracking planes and missiles to predicting the economy. However, HMMs and KFMs are limited in their “expressive power”. Dynamic Bayesian Networks (DBNs) generalize HMMs by allowing the state space to be represented in factored form, instead of as a single discrete random variable. DBNs generalize KFMs by allowing arbitrary probability distributions, not just (unimodal) linear-Gaussian. In this thesis, I will discuss how to represent many different kinds of models as DBNs, how to perform exact and approximate inference in DBNs, and how to learn DBN models from sequential data. In particular, the main novel technical contributions of this thesis are as follows: a way of representing Hierarchical HMMs as DBNs, which enables inference to be done in O(T ) time instead of O(T ), where T is the length of the sequence; an exact smoothing algorithm that takes O(log T ) space instead of O(T ); a simple way of using the junction tree algorithm for online inference in DBNs; new complexity bounds on exact online inference in DBNs; a new deterministic approximate inference algorithm called factored frontier; an analysis of the relationship between the BK algorithm and loopy belief propagation; a way of applying Rao-Blackwellised particle filtering to DBNs in general, and the SLAM (simultaneous localization and mapping) problem in particular; a way of extending the structural EM algorithm to DBNs; and a variety of different applications of DBNs. However, perhaps the main value of the thesis is its catholic presentation of the field of sequential data modelling.

/pdf/dynamic-bayesian-networks-representation-inference-and-yy4fhfxdcj.pdf

Dynamic bayesian networks: representation, inference and learning

A new algorithm for the prediction, filtering, and smoothing of non-Gaussian nonlinear state space models is shown. The algorithm is based on a Monte Carlo method in which successive prediction, filtering (and subsequently smoothing), conditional probability density functions are approximated by many of their realizations. The particular contribution of this algorithm is that it can be applied to a broad class of nonlinear non-Gaussian higher dimensional state space models on the provision that the dimensions of the system noise and the observation noise are relatively low. Several numerical examples are shown.

https://www.jstor.org/stable/pdfplus/1390750.pdf

Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models

Estimation theory has had a tremendous impact on many problem areas over the past two decades. Beginning with its original use in the aerospace industry, its applications can now be found in many different areas such as control and communjcations, power systems, transportation systems, bioengineering, image processing, etc. Along with linear system theory and optimal control, a course in estimation theorycan be found in the graduate system and control curriculum,of most schools in the country. In fact, it is probably one of the most,salable courses as far as employment is concerned. However, despite its economic value and the amount of activities in the field, very few books on estimation theory have appeared recently. This book helps to fill the void in the market and does that in a superb manner. Although the book is called OptimalFiltering, the coverage is restricted to discrete time filtering. A more appropriate title would thus be Optimal Discrete Time ,Filtering. The authors’ decision to concentrate on discrete time f lters is due to “recent technological developments as well as the easier path offered students and instructors.” This is probably a wise move since a thorough treatment of continuous time filtering will require a better knowledge o f stochastic processes than most graduate students or engineers will have. As it stands now, the text requires little background beyond that of linear system theory and probability theory. Written by active researchers, in the area, the book covers the  standard  topics  such  as  Kalman filtering, innovations processes, smoothing, and adaptive and nonlinear estimation. Much of the material in the book has been around for a long time and has been widely used, by practitioners in the area: Some results are more recent. However,-it .has been difficult to locate all of them presented in a n organized manner within a single text. This is especially true of the chapters dealing with the computation aspects and nonlinear and adaptive estimation. After a short introductory chapter, Chapter 2 introduces the mathematical model to be used throughout most of the book. The discrete time Kalman filter is 1 hen presented in Chapter 3, along with some applications. Chapter 4 contains a treatment

http://ieeexplore.ieee.org/iel5/21/4308790/04308806.pdf

Optimal Filtering

Review of "Optimal Filtering" by Brian D. O. Anderson and John B. Moore.

Effect of shoe insole on the dynamics of lower extremities in individuals with leg length discrepancy during walking.

Oscillating and chaotic regimes of honeycomb patterns and dark cavity solitons are studied in a cavity displaying electromagnetically induced transparency. Considering a three-level atomic system in a Λ configuration, transition to chaotically oscillating honeycombs is numerically investigated and their dynamical behavior is detailed for different control parameter values. We show that a branch of oscillating dark cavity solitons coexists with the honeycomb patterns. These oscillating cavity solitons are studied individually and in mutual interaction. In particular, it is shown that mutually coherent oscillating dark cavity solitons can be obtained by choosing an appropriate separation distance. Finally, we introduce a novel regime of chimera states consisting of a multitude of identically and non-locally coupled dark cavity solitons in which regions of in-phase and anti-phase coherence separated by incoherent areas coexist and are stationary. We discuss the unique features of these structures and categorize them as static multihead dual chimera states. We believe that this is the first time that such chimeric states are reported in a photonic system hosting oscillating and chaotic structures. 

Oscillatory and chaotic regimes of patterns and dark cavity solitons in cavities displaying EIT: Static multihead dual chimera states

Temporal cavity solitons in a ring resonator filled with a three-level medium displaying electromagnetically induced transparency require less input power, have better shapes and produce optimal frequency combs than their two-level media counterparts.

Mansour Eslami

Papers

Optimal Filtering

Review of "Optimal Filtering" by Brian D. O. Anderson and John B. Moore.

Effect of shoe insole on the dynamics of lower extremities in individuals with leg length discrepancy during walking.

Oscillatory and chaotic regimes of patterns and dark cavity solitons in cavities displaying EIT: Static multihead dual chimera states

Temporal cavity solitons due to electromagnetically induced transparency