Increasingly, for many application areas, it is becoming important to include elements of nonlinearity and non-Gaussianity in order to model accurately the underlying dynamics of a physical system. Moreover, it is typically crucial to process data on-line as it arrives, both from the point of view of storage costs as well as for rapid adaptation to changing signal characteristics. In this paper, we review both optimal and suboptimal Bayesian algorithms for nonlinear/non-Gaussian tracking problems, with a focus on particle filters. Particle filters are sequential Monte Carlo methods based on point mass (or "particle") representations of probability densities, which can be applied to any state-space model and which generalize the traditional Kalman filtering methods. Several variants of the particle filter such as SIR, ASIR, and RPF are introduced within a generic framework of the sequential importance sampling (SIS) algorithm. These are discussed and compared with the standard EKF through an illustrative example.

/pdf/a-tutorial-on-particle-filters-for-online-nonlinear-non-1rzkwgk2pp.pdf

A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking

Any series of observations ordered along a single dimension, such as time, may be thought of as a time series. The emphasis in time series analysis is on studying the dependence among observations at different points in time. What distinguishes time series analysis from general multivariate analysis is precisely the temporal order imposed on the observations. Many economic variables, such as GNP and its components, price indices, sales, and stock returns are observed over time. In addition to being interested in the contemporaneous relationships among such variables, we are often concerned with relationships between their current and past values, that is, relationships over time.

Time Series Analysis

计量经济分析 = Econometric analysis

Monte Carlo methods are revolutionizing the on-line analysis of data in fields as diverse as financial modeling, target tracking and computer vision. These methods, appearing under the names of bootstrap filters, condensation, optimal Monte Carlo filters, particle filters and survival of the fittest, have made it possible to solve numerically many complex, non-standard problems that were previously intractable. This book presents the first comprehensive treatment of these techniques, including convergence results and applications to tracking, guidance, automated target recognition, aircraft navigation, robot navigation, econometrics, financial modeling, neural networks, optimal control, optimal filtering, communications, reinforcement learning, signal enhancement, model averaging and selection, computer vision, semiconductor design, population biology, dynamic Bayesian networks, and time series analysis. This will be of great value to students, researchers and practitioners, who have some basic knowledge of probability. Arnaud Doucet received the Ph. D. degree from the University of Paris-XI Orsay in 1997. From 1998 to 2000, he conducted research at the Signal Processing Group of Cambridge University, UK. He is currently an assistant professor at the Department of Electrical Engineering of Melbourne University, Australia. His research interests include Bayesian statistics, dynamic models and Monte Carlo methods. Nando de Freitas obtained a Ph.D. degree in information engineering from Cambridge University in 1999. He is presently a research associate with the artificial intelligence group of the University of California at Berkeley. His main research interests are in Bayesian statistics and the application of on-line and batch Monte Carlo methods to machine learning. Neil Gordon obtained a Ph.D. in Statistics from Imperial College, University of London in 1993. He is with the Pattern and Information Processing group at the Defence Evaluation and Research Agency in the United Kingdom. His research interests are in time series, statistical data analysis, and pattern recognition with a particular emphasis on target tracking and missile guidance.

/pdf/sequential-monte-carlo-methods-in-practice-3kfxeebn8e.pdf

Sequential Monte Carlo methods in practice

In this article, we present an overview of methods for sequential simulation from posterior distributions. These methods are of particular interest in Bayesian filtering for discrete time dynamic models that are typically nonlinear and non-Gaussian. A general importance sampling framework is developed that unifies many of the methods which have been proposed over the last few decades in several different scientific disciplines. Novel extensions to the existing methods are also proposed. We show in particular how to incorporate local linearisation methods similar to those which have previously been employed in the deterministic filtering literatures these lead to very effective importance distributions. Furthermore we describe a method which uses Rao-Blackwellisation in order to take advantage of the analytic structure present in some important classes of state-space models. In a final section we develop algorithms for prediction, smoothing and evaluation of the likelihood in dynamic models.

/pdf/on-sequential-monte-carlo-sampling-methods-for-bayesian-3bit26usft.pdf

On sequential Monte Carlo sampling methods for Bayesian filtering

Limit distribution results on realized power variation, that is, sums of absolute powers of increments of a process, are derived for certain types of semimartingale with continuous local martingale component, in particular for a class of flexible stochastic volatility models. The theory covers, for example, the cases of realized volatility and realized absolute variation. Such results should be helpful in, for example, the analysis of volatility models using high-frequency information.

/pdf/realized-power-variation-and-stochastic-volatility-models-7gj361nz26.pdf

Realized power variation and stochastic volatility models

This paper reviews some recent work in which Lprocesses are used to model and ana- lyse time series from financial econometrics. A main feature of the paper is the use of positive Ornstein- Uhlenbeck (OU) type processes inside stochastic volatility processes. The basic probability theory asso- ciated with such models is discussed in some detail.

/pdf/modelling-by-levy-processes-for-financial-econometrics-4qperv9jxa.pdf

Modelling by Lévy Processes for Financial Econometrics

Building models for high dimensional portfolios is important in risk management and asset allocation. Here we propose a novel and fast way of estimating models of time-varying covariances that overcome an undiagnosed incidental parameter problem which has troubled existing methods when applied to hundreds or even thousands of assets. Indeed we can handle the case where the cross-sectional dimension is larger than the time series one. The theory of this new strategy is developed in some detail, allowing formal hypothesis testing to be carried out on these models. Simulations are used to explore the performance of this inference strategy while empirical examples are reported which show the strength of this method. The out of sample hedging performance of various models estimated using this method are compared.

https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID3296215_code1907585.pdf?abstractid=1354497&mirid=1

Fitting Vast Dimensional Time-Varying Covariance Models

Suppose we wish to carry out likelihood based inference but we solely have an unbiased simulation based estimator of the likelihood. We note that unbiasedness is enough when the estimated likelihood is used inside a Metropolis-Hastings algorithm. This result has recently been introduced in statistics literature by Andrieu, Doucet, and Holenstein (2007) and is perhaps surprising given the celebrated results on maximum simulated likelihood estimation. Bayesian inference based on simulated likelihood can be widely applied in microeconomics, macroeconomics and flnancial econometrics. One way of generating unbiased estimates of the likelihood is by the use of a particle fllter. We illustrate these methods on four problems in econometrics, producing rather generic methods. Taken together, these methods imply that if we can simulate from an economic model we can carry out likelihood based inference using its simulations.

/pdf/bayesian-inference-based-only-on-simulated-likelihood-4gs5zstnkk.pdf

Bayesian inference based only on simulated likelihood: particle filter analysis of dynamic economic models

Consider a semimartingale of the form Y_{t}=Y_0+\int _0^{t}a_{s}ds+\int _0^{t}_{s-} dW_{s}, where a is a locally bounded predictable process and (the volatility) is an adapted right - continuous process with left limits and W is a Brownian motion. We define the realised bipower variation process V(Y;r,s)_{t}^n=n^{((r+s)/2)-1} \sum_{i=1}^{[nt]}|Y_{(i/n)}-Y_{((i-1)/n)}|^{r}|Y_{((i+1)/n)}-Y_{(i/n)}|^{s}, where r and s are nonnegative reals with r+s>0. We prove that V(Y;r,s)_{t}n converges locally uniformly in time, in probability, to a limiting process V(Y;r,s)_{t} (the bipower variation process). If further is a possibly discontinuous semimartingale driven by a Brownian motion which may be correlated with W and by a Poisson random measure, we prove a central limit theorem, in the sense that \sqrt(n) (V(Y;r,s)^n-V(Y;r,s)) converges in law to a process which is the stochastic integral with respect to some other Brownian motion W', which is independent of the driving terms of Y and \sigma. We also provide a multivariate version of these results.

Neil Shephard

Papers

Realized power variation and stochastic volatility models

Modelling by Lévy Processes for Financial Econometrics

Fitting Vast Dimensional Time-Varying Covariance Models

Bayesian inference based only on simulated likelihood: particle filter analysis of dynamic economic models

A Central Limit Theorem for Realised Power and Bipower Variations of Continuous Semimartingales