Expectation–maximization algorithm

About: Expectation–maximization algorithm is a research topic. Over the lifetime, 11823 publications have been published within this topic receiving 528693 citations. The topic is also known as: EM algorithm & Expectation Maximization.

Generative model-based clustering of directional data

Abstract: High dimensional directional data is becoming increasingly important in contemporary applications such as analysis of text and gene-expression data. A natural model for multi-variate directional data is provided by the von Mises-Fisher (vMF) distribution on the unit hypersphere that is analogous to the multi-variate Gaussian distribution in Rd. In this paper, we propose modeling complex directional data as a mixture of vMF distributions. We derive and analyze two variants of the Expectation Maximization (EM) framework for estimating the parameters of this mixture. We also propose two clustering algorithms corresponding to these variants. An interesting aspect of our methodology is that the spherical kmeans algorithm (kmeans with cosine similarity) can be shown to be a special case of both our algorithms. Thus, modeling text data by vMF distributions lends theoretical validity to the use of cosine similarity which has been widely used by the information retrieval community. As part of experimental validation, we present results on modeling high-dimensional text and gene-expression data as a mixture of vMF distributions. The results indicate that our approach yields superior clusterings especially for difficult clustering tasks in high-dimensional spaces.

New finite-dimensional filters for parameter estimation of discrete-time linear Gaussian models

Abstract: The authors derive a new class of finite-dimensional recursive filters for linear dynamical systems. The Kalman filter is a special case of their general filter. Apart from being of mathematical interest, these new finite-dimensional filters can be used with the expectation maximization (EM) algorithm to yield maximum likelihood estimates of the parameters of a linear dynamical system. Important advantages of their filter-based EM algorithm compared with the standard smoother-based EM algorithm include: 1) substantially reduced memory requirements, and 2) ease of parallel implementation on a multiprocessor system. The algorithm has applications in multisensor signal enhancement of speech signals and also econometric modeling.

A hierarchical approach to multivariate spatial modeling and prediction

Abstract: We propose a hierarchical model for multivariate spatial modeling and prediction under which one specifies a joint distribution for a multivariate spatial process indirectly through specification of simpler conditional models. This approach is similar to standard methods known as cokriging and kriging with external drift,' but avoids some of the inherent difficulties in these two approaches including specification of valid joint covariance models and restriction to exhaustively sampled covariates. Moreover, both existing approaches can be formulated in this hierarchical framework. The hierarchical approach is ideally suited for, but not restricted for use in, situations in which known cause/effect' relationships exist. Because the hierarchical approach models dependence between variables in conditional means, as opposed to cross-covariances, very complicated relationships are more easily parameterized. We suggest an iterative estimation procedure that combines generalized least squares with imputation of missing values using the best linear unbiased predictor. An example is given that involves prediction of a daily ozone summary from maximum daily temperature in the Midwest.

Dictionary-based stochastic expectation-maximization for SAR amplitude probability density function estimation

Abstract: In remotely sensed data analysis, a crucial problem is represented by the need to develop accurate models for the statistics of the pixel intensities. This paper deals with the problem of probability density function (pdf) estimation in the context of synthetic aperture radar (SAR) amplitude data analysis. Several theoretical and heuristic models for the pdfs of SAR data have been proposed in the literature, which have been proved to be effective for different land-cover typologies, thus making the choice of a single optimal parametric pdf a hard task, especially when dealing with heterogeneous SAR data. In this paper, an innovative estimation algorithm is described, which faces such a problem by adopting a finite mixture model for the amplitude pdf, with mixture components belonging to a given dictionary of SAR-specific pdfs. The proposed method automatically integrates the procedures of selection of the optimal model for each component, of parameter estimation, and of optimization of the number of components by combining the stochastic expectation-maximization iterative methodology with the recently developed "method-of-log-cumulants" for parametric pdf estimation in the case of nonnegative random variables. Experimental results on several real SAR images are reported, showing that the proposed method accurately models the statistics of SAR amplitude data.

Parametric fractional imputation for missing data analysis

Abstract: Under a parametric model for missing data, the EM algorithm is a popular tool for flnding the maximum likelihood estimates (MLE) of the parameters of the model. Imputation, when carefully done, can be used to facilitate the parameter estimation by applying the complete-sample estimators to the imputed dataset. The basic idea is to generate the imputed values from the conditional distribution of the missing data given the observed data. Multiple imputation is a Bayesian approach to generate the imputed values from the conditional distribution. In this article, parametric fractional imputation is proposed as a parametric approach for generating imputed values. Using fractional weights, the E-step of the EM algorithm can be approximated by the weighted mean of the imputed data likelihood where the fractional weights are computed from the current value of the parameter estimates. Some computational e‐ciency can be achieved using the idea of importance sampling in the Monte Carlo approximation of the conditional expectation. The resulting estimator of the specifled parameters will be identical to the MLE under missing data if the fractional weights are adjusted using a calibration step. The proposed imputation method provides e‐cient parameter estimates for the model parameters specifled and also provides reasonable estimates for parameters that are not part of the imputation model, for example domain means. Thus, the proposed imputation method is a useful tool for general-purpose data analysis. Variance estimation is covered and results from a limited simulation study are presented.