An overview of statistical and information-theoretic aspects of hidden Markov processes (HMPs) is presented. An HMP is a discrete-time finite-state homogeneous Markov chain observed through a discrete-time memoryless invariant channel. In recent years, the work of Baum and Petrie (1966) on finite-state finite-alphabet HMPs was expanded to HMPs with finite as well as continuous state spaces and a general alphabet. In particular, statistical properties and ergodic theorems for relative entropy densities of HMPs were developed. Consistency and asymptotic normality of the maximum-likelihood (ML) parameter estimator were proved under some mild conditions. Similar results were established for switching autoregressive processes. These processes generalize HMPs. New algorithms were developed for estimating the state, parameter, and order of an HMP, for universal coding and classification of HMPs, and for universal decoding of hidden Markov channels. These and other related topics are reviewed.

http://www.isib.cnr.it/control/finesso/Ephraim_hmm.pdf

Hidden Markov processes

Nonlinear stochastic dynamical systems are widely used to model systems across the sciences and engineering. Such models are natural to formulate and can be analyzed mathematically and numerically. However, difficulties associated with inference from time-series data about unknown parameters in these models have been a constraint on their application. We present a new method that makes maximum likelihood estimation feasible for partially-observed nonlinear stochastic dynamical systems (also known as state-space models) where this was not previously the case. The method is based on a sequence of filtering operations which are shown to converge to a maximum likelihood parameter estimate. We make use of recent advances in nonlinear filtering in the implementation of the algorithm. We apply the method to the study of cholera in Bangladesh. We construct confidence intervals, perform residual analysis, and apply other diagnostics. Our analysis, based upon a model capturing the intrinsic nonlinear dynamics of the system, reveals some effects overlooked by previous studies.

https://www.pnas.org/content/pnas/103/49/18438.full.pdf

Inference for nonlinear dynamical systems

We evaluate a method for fitting models to time series of population abun- dances that incorporates both process noise and observation error in a likelihood framework. The method follows the probability logic of the Kalman filter, but whereas the Kalman filter applies to linear, Gaussian systems, we implement the full probability calculations numerically so that any nonlinear, non-Gaussian model can be used. We refer to the method as the "numerically integrated state-space (NISS) method" and compare it to two common methods used to analyze nonlinear time series in ecology: least squares with only process noise (LSPN) and least squares with only observation error (LSOE). We compare all three methods by fitting Beverton-Holt and Ricker models to many replicate model-generated time series of length 20 with several parameter choices. For the Ricker model we chose parameters for which the deterministic part of the model produces a stable equilibrium, a two-cycle, or a four-cycle. For each set of parameters we used three process-noise and observation-error scenarios: large standard deviation (0.2) for both, and large for one but small (0.05) for the other. The NISS method had lower estimator bias and variance than the other methods in nearly all cases. The only exceptions were for the Ricker model with stable-equilibrium parameters, in which case the LSPN and LSOE methods has lower bias when noise variances most closely met their assumptions. For the Beverton-Holt model, the NISS method was much less biased and more precise than the other methods. We also evaluated the utility of each method for model selection by fitting simulated data to both models and using information criteria for selection. The NISS and LSOE methods showed a strong bias toward selecting the Ricker over the Beverton-Holt, even when data were generated with the Beverton-Holt. It remains unclear whether the LSPN method is generally superior for model selection or has fortuitously better biases in this particular case. These results suggest that information criteria are best used with caution for nonlinear population models with short time series. Finally we evaluated the convergence of likelihood ratios to theoretical asymptotic distributions. Agreement with asymptotic distributions was very good for stable-point Rick- er parameters, less accurate for two-cycle and four-cycle Ricker parameters, and least accurate for the Beverton-Holt model. The numerically integrated state-space method has a number of advantages over least squares methods and offers a useful tool for connecting models and data and ecology.

/pdf/fitting-population-models-incorporating-process-noise-and-563cajs1m8.pdf

Fitting population models incorporating process noise and observation error

1. Concepts, models and definitions 2. Nonlinear models in economic theory 3. Parametric nonlinear models 4. The nonparametric approach 5. Parametric linearity tests 6. Testing parameter constancy 7. Nonparametric specification tests 8. Conditional heteroskedasticity 9. State space models 10. Nonparametric models 11. Nonlinear and nonstationary models 12. Estimating parametric models 13. Basic nonparametric estimates 14. Forecasting from nonlinear models 15. Nonlinear impulse responses 16. Building nonlinear models 17. Other topics

/pdf/modelling-nonlinear-economic-time-series-vydxvd2m8n.pdf

Modelling nonlinear economic time series

An autoregressive process with Markov regime is an autoregressive process for which the regression function at each time point is given by a nonobservable Markov chain. In this paper we consider the asymptotic properties of the maximum likelihood estimator in a possibly nonstationary process of this kind for which the hidden state space is compact but not necessarily finite. Consistency and asymptotic normality are shown to follow from uniform exponential forgetting of the initial distribution for the hidden Markov chain conditional on the observations.

/pdf/asymptotic-properties-of-the-maximum-likelihood-estimator-in-2jst3jk5qy.pdf

Asymptotic properties of the maximum likelihood estimator in autoregressive models with Markov regime

State space models is a very general class of time series models capable of modelling dependent observations in a natural and interpretable way. Inference in such models has been studied by Bickel, Ritov and Ryden, who consider hidden Markov models, which are special kinds of state space models, and prove that the maximum likelihood estimator is asymptotically normal under mild regularity conditions. In this paper we generalize the results of Bickel, Ritov and Ryden to state space models, where the latent process is a continuous state Markov chain satisfying regularity conditions, which are fulfilled if the latent process takes values in a compact space.

/pdf/asymptotic-normality-of-the-maximum-likelihood-estimator-in-hm3xrhohee.pdf

Niels Væver Petersen

Papers

Asymptotic normality of the maximum likelihood estimator in state space models