Increasingly, for many application areas, it is becoming important to include elements of nonlinearity and non-Gaussianity in order to model accurately the underlying dynamics of a physical system. Moreover, it is typically crucial to process data on-line as it arrives, both from the point of view of storage costs as well as for rapid adaptation to changing signal characteristics. In this paper, we review both optimal and suboptimal Bayesian algorithms for nonlinear/non-Gaussian tracking problems, with a focus on particle filters. Particle filters are sequential Monte Carlo methods based on point mass (or "particle") representations of probability densities, which can be applied to any state-space model and which generalize the traditional Kalman filtering methods. Several variants of the particle filter such as SIR, ASIR, and RPF are introduced within a generic framework of the sequential importance sampling (SIS) algorithm. These are discussed and compared with the standard EKF through an illustrative example.

/pdf/a-tutorial-on-particle-filters-for-online-nonlinear-non-1rzkwgk2pp.pdf

A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking

Any series of observations ordered along a single dimension, such as time, may be thought of as a time series. The emphasis in time series analysis is on studying the dependence among observations at different points in time. What distinguishes time series analysis from general multivariate analysis is precisely the temporal order imposed on the observations. Many economic variables, such as GNP and its components, price indices, sales, and stock returns are observed over time. In addition to being interested in the contemporaneous relationships among such variables, we are often concerned with relationships between their current and past values, that is, relationships over time.

Time Series Analysis

计量经济分析 = Econometric analysis

Monte Carlo methods are revolutionizing the on-line analysis of data in fields as diverse as financial modeling, target tracking and computer vision. These methods, appearing under the names of bootstrap filters, condensation, optimal Monte Carlo filters, particle filters and survival of the fittest, have made it possible to solve numerically many complex, non-standard problems that were previously intractable. This book presents the first comprehensive treatment of these techniques, including convergence results and applications to tracking, guidance, automated target recognition, aircraft navigation, robot navigation, econometrics, financial modeling, neural networks, optimal control, optimal filtering, communications, reinforcement learning, signal enhancement, model averaging and selection, computer vision, semiconductor design, population biology, dynamic Bayesian networks, and time series analysis. This will be of great value to students, researchers and practitioners, who have some basic knowledge of probability. Arnaud Doucet received the Ph. D. degree from the University of Paris-XI Orsay in 1997. From 1998 to 2000, he conducted research at the Signal Processing Group of Cambridge University, UK. He is currently an assistant professor at the Department of Electrical Engineering of Melbourne University, Australia. His research interests include Bayesian statistics, dynamic models and Monte Carlo methods. Nando de Freitas obtained a Ph.D. degree in information engineering from Cambridge University in 1999. He is presently a research associate with the artificial intelligence group of the University of California at Berkeley. His main research interests are in Bayesian statistics and the application of on-line and batch Monte Carlo methods to machine learning. Neil Gordon obtained a Ph.D. in Statistics from Imperial College, University of London in 1993. He is with the Pattern and Information Processing group at the Defence Evaluation and Research Agency in the United Kingdom. His research interests are in time series, statistical data analysis, and pattern recognition with a particular emphasis on target tracking and missile guidance.

/pdf/sequential-monte-carlo-methods-in-practice-3kfxeebn8e.pdf

Sequential Monte Carlo methods in practice

In this article, we present an overview of methods for sequential simulation from posterior distributions. These methods are of particular interest in Bayesian filtering for discrete time dynamic models that are typically nonlinear and non-Gaussian. A general importance sampling framework is developed that unifies many of the methods which have been proposed over the last few decades in several different scientific disciplines. Novel extensions to the existing methods are also proposed. We show in particular how to incorporate local linearisation methods similar to those which have previously been employed in the deterministic filtering literatures these lead to very effective importance distributions. Furthermore we describe a method which uses Rao-Blackwellisation in order to take advantage of the analytic structure present in some important classes of state-space models. In a final section we develop algorithms for prediction, smoothing and evaluation of the likelihood in dynamic models.

/pdf/on-sequential-monte-carlo-sampling-methods-for-bayesian-3bit26usft.pdf

On sequential Monte Carlo sampling methods for Bayesian filtering

This paper is concerned with the fitting and comparison of high dimensional multivariate time series models with time varying correlations. The models considered here combine features of the classical factor model with those of the univariate stochastic volatility model. Specifically, a set of unobserved time-dependent factors, along with an associated loading matrix, are used to model the contemporaneous correlation while, conditioned on the factors, the noise in each factor and each series is assumed to follow independent three-parameter univariate stochastic volatility processes. A complete analysis of these models, and its special cases, is developed that encompasses estimation, filtering and model choice. The centerpieces of our estimation algorithm (which relies on MCMC methods) is (1) a reduced blocking scheme for sampling the free elements of the loading matrix and the factors and (2) a special method for sampling the parameters of the univariate SV process. The sampling of the loading matrix (containing typically many hundreds of parameters) is done via a highly tuned Metropolis-Hastings step. The resulting algorithm is completely scalable in terms of series and factors and very simulation-efficient. We also provide methods for estimating the log-likelihood function and the filtered values of the time-varying volatilities and correlations. We pay special attention to the problem of comparing one version of the model with another and for determining the number of factors. For this purpose we use MCMC methods to find the marginal likelihood and associated Bayes factors of each fitted model. In sum, these procedures lead to the first unified and practical likelihood based analysis of truly high dimensional models of stochastic volatility. We apply our methods in detail to two datasets. The first is the return vector on 20 exchange rates against the US Dollar. The second is the return vector on 40 common stocks quoted on the New York Stock Exchange.

Analysis of High Dimensional Multivariate Stochastic Volatility Models

In panel experiments, we randomly assign units to different interventions, measuring their outcomes, and repeating the procedure in several periods. Using the potential outcomes framework, we define finite population dynamic causal effects that capture the relative effectiveness of alternative treatment paths. For a rich class of dynamic causal effects, we provide a nonparametric estimator that is unbiased over the randomization distribution and derive its finite population limiting distribution as either the sample size or the duration of the experiment increases. We develop two methods for inference: a conservative test for weak null hypotheses and an exact randomization test for sharp null hypotheses. We further analyze the finite population probability limit of linear fixed effects estimators. These commonly-used estimators do not recover a causally interpretable estimand if there are dynamic causal effects and serial correlation in the assignments, highlighting the value of our proposed estimator.

Panel Experiments and Dynamic Causal Effects: A Finite Population Perspective

This paper introduces a new continuous-time framework for modelling serially correlated count and integer-valued data. The key component in our new model is the class of integer-valued trawl processes, which are serially correlated, stationary, infinitely divisible processes. We analyse the probabilistic properties of such processes in detail and, in addition, study volatility modulation and multivariate extensions within the new modelling framework. Moreover, we describe how the parameters of a trawl process can be estimated and obtain promising estimation results in our simulation study. Finally, we apply our new modelling framework to high-frequency financial data.

/pdf/integer-valued-trawl-processes-a-class-of-stationary-335ils4dbm.pdf

Integer‐valued Trawl Processes: A Class of Stationary Infinitely Divisible Processes

A stochastic variance model may be estimated by quasi-maximum likelihood procedure by transforming to a linear state space form. The properties of observations corrected for heteroscedasticity can be derived. A model with explanatory variables can be handled by correcting the observations for heteroscedasticity after estimating a stochastic variance model from the OLS residuals and then constructing a feasible GLS estimator. A model with stochastic variance, or standard deviation, as an explanatory variable can also be formulated. The paper explores the properties of these procedures and shows how they may be used as part of a model specification strategy/ It is argued that the approach is relatively robust since distributions need not be specified for the disturbances.

Estimation and Testing of Stochastic Variance Models

We investigate the properties of the composite likelihood (CL) method for (T × NT ) GARCH panels. The defining feature of a GARCH panel with time- series length T is that, while nuisance parameters are allowed to vary across NT series, other parameters of interest are assumed to be common. CL pools informa- tion across the panel instead of using information available in a single series only. Simulations and empirical analysis illustrate that when T is reasonably large CL performs well. However, due to the presence of nuisance parameters, CL is subject to the "incidental parameter" problem for small T.

/pdf/nuisance-parameters-composite-likelihoods-and-a-panel-of-1tqbji8450.pdf

Neil Shephard

Papers

Analysis of High Dimensional Multivariate Stochastic Volatility Models

Panel Experiments and Dynamic Causal Effects: A Finite Population Perspective

Integer‐valued Trawl Processes: A Class of Stationary Infinitely Divisible Processes

Estimation and Testing of Stochastic Variance Models

Nuisance parameters, composite likelihoods and a panel of garch models