Showing papers on "Expectation–maximization algorithm published in 1990"

A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms

Abstract: The first part of this article presents the Monte Carlo implementation of the E step of the EM algorithm. Given the current guess to the maximizer of the posterior distribution, latent data patterns are generated from the conditional predictive distribution. The expected value of the augmented log-posterior is then updated as a mixture of augmented log-posteriors, mixed over the generated latent data patterns (multiple imputations). In the M step of the algorithm, this mixture is maximized to obtain the update to the maximizer of the observed posterior. The gradient and Hessian of the observed log posterior are also expressed as mixtures, mixed over the multiple imputations. The relation between the Monte Carlo EM (MCEM) algorithm and the data augmentation algorithm is noted. Two modifications to the MCEM algorithm (the poor man's data augmentation algorithms), which allow for the calculation of the entire posterior, are then presented. These approximations serve as diagnostics for the validity o...

Bayesian reconstructions from emission tomography data using a modified EM algorithm

Abstract: A novel method of reconstruction from single-photon emission computerized tomography data is proposed. This method builds on the expectation-maximization (EM) approach to maximum likelihood reconstruction from emission tomography data, but aims instead at maximum posterior probability estimation, which takes account of prior belief about smoothness in the isotope concentration. A novel modification to the EM algorithm yields a practical method. The method is illustrated by an application to data from brain scans. >

Convergence of EM image reconstruction algorithms with Gibbs smoothing

Abstract: P.J. Green has defined an OSL (one-step late) algorithm that retains the E-step of the EM algorithm (for image reconstruction in emission tomography) but provides an approximate solution to the M-step. Further modifications of the OSL algorithm guarantee convergence to the unique maximum of the log posterior function. Convergence is proved under a specific set of sufficient conditions. Several of these conditions concern the potential function of the Gibb's prior, and a number of candidate potential functions are identified. Generalization of the OSL algorithm to transmission tomography is also considered. >

On Use of the EM Algorithm for Penalized Likelihood Estimation

Abstract: SUMMARY The EM algorithm is a popular approach to maximum likelihood estimation but has not been much used for penalized likelihood or maximum a posteriori estimation This paper discusses properties of the EM algorithm in such contexts, concentrating on rates of conver- gence, and presents an alternative that is usually more practical and converges at least as quickly The EM algorithm is a general approach to maximum likelihood estimation, rather than a specific algorithm Dempster et al (1977) discussed the method and derived basic properties, demonstrating that a variety of procedures previously developed rather informally could be unified The common strand to problems where the approach is applicable is a notion of 'incomplete data'; this includes the conventional sense of 'missing data' but is much broader than that The EM algorithm demon- strates its strength in situations where some hypothetical experiment yields data from which estimation is particularly convenient and economical: the 'incomplete' data actually at hand are regarded as observable functions of these 'complete' data The resulting algorithms, while usually slow to converge, are often extremely simple and remain practical in large problems where no other approaches may be feasible Dempster et al (1977) briefly refer to the use of the same approach to the problem of finding the posterior mode (maximum a posteriori estimate) in a Bayesian estima-

Incomplete Data in Generalized Linear Models

Abstract: This article examines incomplete data for the class of generalized linear models, in which incompleteness is due to partially missing covariates on some observations. Under the assumption that the missing data are missing at random, it is shown that the E step of the EM algorithm for any generalized linear model can be expressed as a weighted complete data log-likelihood when the unobserved covariates are assumed to come from a discrete distribution with finite range. Expressing the E step in this manner allows for a straightforward maximization in the M step, thus leading to maximum likelihood estimates (MLE's) for the parameters. Asymptotic variances of the MLE's are also derived, and results are illustrated with two examples.

A smoothed EM approach to indirect estimation problems, with particular reference to stereology and emission tomography

Abstract: There are many practical problems where the observed data are not drawn directly from the density g of real interest, but rather from another distribution derived from g by the application of an integral operator. The estimation of g then entails both statistical and numerical difficulties. A natural statistical approach is by maximum likelihood, conveniently implemented using the EM algorithm, but this provides unsatisfactory reconstructions of g. In this paper, we modify the maximum likelihood-EM approach by introducing a simple smoothing step at each EM iteration. In our experience, this algorithm converges in relatively few iterations to good estimates of g that do not depend on the choice of starting configuration. Some theoretical background is given that relates this smoothed EM algorithm to a maximum penalized likelihood approach. Two applications are considered in detail. The first is the classical stereology problem of determining particle size distributions from data collected on a plane section through a composite medium. The second concerns the recovery of the structure of a section of the human body from external observations obtained by positron emission tomography; for this problem, we also suggest several technical improvements on existing methodology.

Identification and restoration of noisy blurred images using the expectation-maximization algorithm

Abstract: A maximum-likelihood approach to the blur identification problem is presented. The expectation-maximization algorithm is proposed to optimize the nonlinear likelihood function in an efficient way. In order to improve the performance of the identification algorithm, low-order parametric image and blur models are incorporated into the identification method. The resulting iterative technique simultaneously identifies and restores noisy blurred images. >

Maximum-likelihood narrow-band direction finding and the EM algorithm

Abstract: Generalized EM (expectation-maximization) algorithms have been derived for the maximum-likelihood estimation of the direction-of-arrival of multiple narrowband signals in noise. Both deterministic and stochastic signal models are considered. The algorithm for the deterministic model yields estimates of the signal amplitudes, while that for the stochastic model yields estimates of the powers of the signal. Both algorithms have the properties that their limit points are stable and satisfy the necessary maximizer conditions for maximum-likelihood estimators. It is shown via simulation that the maximum-likelihood method allows for the resolution of the directions-of-arrival of signals at angular separation and noise levels for which other high-resolution methods will not work. Algorithm convergence does depend on initial conditions; however, convergence to a global maximum has been observed in simulation when the initial estimates are within a significant fraction if one beamwidth (componentwise) of this maximum. Simulations also show that the deterministic model has a significant impact on the angle estimator performance. >

Estimation Procedures for Structural Time Series Models

Abstract: A univariate structural time series model based on the traditional decomposition into trend, seasonal and irregular components is defined. A number of methods of computing maximum likelihood estimators are then considered. These include direct maximization of various time domain likelihood function. The asymptotic properties of the estimators are given and a comparison between the various methods in terms of computational efficiency and accuracy is made. The methods are then extended to models with explanatory variables. Ktv WORDS Structural time series model Forecasting Kalman filter Stochastic trend Unobserved components model EM algorithm

Extended maximum likelihood

Abstract: An account is given of the method of extended maximum likelihood. This differs from the standard method of maximum likelihood in that the normalisation of the probability distribution function is allowed to vary. It is thus applicable to problems in which the number of samples obtained is itself a relevant measurement. If the function is such that its size and shape can be independently varied, then the estimates given by the extended method are identical to the standard maximum likelihood estimators, though the errors require care of interpretation. If the function does not have this property, then extended maximum likelihood can give better results.

Image identification and restoration based on the expectation-maximization algorithm

Abstract: In this paper, the problem of identifying the image and blur parameters and restoring a noisy blurred image is addressed. Specifying the blurring process by its point spread function (PSF), the blur identification problem is formulated as the maximum likelihood estimation (MLE) of the PSF. Modeling the original image and the additive noise as zeromean Gaussian processes, the MLE of their covariance matrices is also computed. An iterative approach, called the EM (expectation-maximization) algorithm, is used to find the maximum likelihood estimates ofthe relevant unknown parameters. In applying the EM algorithm, the original image is chosen to be part of the complete data; its estimate is computed in the E-step of the EM iterations and represents the restored image. Two algorithms for identification/restoration, based on two different choices of complete data, are derived and compared. Simultaneous blur identification and restoration is performed by the first algorithm, while the identification of the blur results from a separate minimization in the second algorithm. Experiments with simulated and photographically blurred images are shown.

Three-dimensional maximum-likelihood reconstruction for an electronically collimated single-photon-emission imaging system

Abstract: A three-dimensional maximum-likelihood reconstruction method is presented for a prototype electronically collimated single-photon-emission system. The electronically collimated system uses a gamma camera fronted by an array of germanium detectors to detect gamma-ray emissions from a distributed radioisotope source. In this paper we demonstrate that optimal iterative three-dimensional reconstruction approaches can be feasibly applied to emission imaging systems that have highly complex spatial sampling patterns and that generate extremely large numbers of data values. A probabilistic factorization of the system matrix that reduces the computation by several orders of magnitude is derived. We demonstrate a dramatic increase in the convergence speed of the expectation maximization algorithm by sequentially iterating over particular subsets of the data. This result is also applicable to other emission imaging systems.

Maximum likelihood estimates of polar motion parameters

Abstract: Two estimators developed by Jeffreys (1940, 1968) are described and used in conjunction with polar-motion data to determine the frequency (Fc) and quality factor (Qc) of the Chandler wobble. Data are taken from a monthly polar-motion series, satellite laser-ranging results, and optical astrometry and intercompared for use via interpolation techniques. Maximum likelihood arguments were employed to develop the estimators, and the assumption that polar motion relates to a Gaussian random process is assessed in terms of the accuracies of the estimators. The present results agree with those from Jeffreys' earlier study but are inconsistent with the later estimator; a Monte Carlo evaluation of the estimators confirms that the 1968 method is more accurate. The later estimator method shows good performance because the Fourier coefficients derived from the data have signal/noise levels that are superior to those for an individual datum. The method is shown to be valuable for general spectral-analysis problems in which isolated peaks must be analyzed from noisy data.

Maximum Likelihood Estimates of Symmetric Stable Distribution Parameters

Abstract: A method is developed to directly obtain maximum likelihood estimates of symmetric stable distribution parameters. This is a difficult estimation problem since the likelihood function is expressed as an integral. The estimation routine is tested on a Monte Carlo sample and produces reasonable estimates.

An approximation to maximum likelihood estimates in reduced models

Abstract: SUMMARY An approximation to the maximum likelihood estimates of the parameters in a model can be obtained from the corresponding estimates and information matrices in an extended model, i.e. a model with additional parameters. The approximation is close provided that the data are consistent with the first model. Applications are described to log linear models for discrete data, to models for multivariate normal distributions with special covariance matrices and to mixed discrete-continuous models.

Statistical stopping criteria for iterative maximum likelihood reconstruction of emission images

Abstract: Emission tomography involves the reconstruction of images representing the radionudide concentration throughout a patient's body. Maximum likelihood estimation yields excessively noisy images due to the non-uniqueness of the solution and the noise in the emission data. Many investigators have noted that better image reconstructions are obtained by stopping an iterative maximum likelihood algorithm early, well before convergence. This paper explains why better results can be obtained by suchan approach. A statistical criterion for determining an optimal stopping point developed in an earlier paper is compared with two statistics derived and applied here.

A latent trait model for dichotomous choice data

Abstract: The PARELLA model is a probabilistic parallelogram model that can be used for the measurement of latent attitudes or latent preferences. The data analyzed are the dichotomous responses of persons to stimuli, with a one (zero) indicating agreement (disagreement) with the content of the stimulus. The model provides a unidimensional representation of persons and items. The response probabilities are a function of the distance between person and stimulus: the smaller the distance, the larger the probability that a person will agree with the content of the stimulus. An estimation procedure based on expectation maximization and marginal maximum likelihood is developed and the quality of the resulting parameter estimates evaluated.

On the maximum likelihood estimation of the location and scale parameters of exponential distribution based on multiply Type II censored samples

Abstract: The maximum likelihood (ML) estimation of the location and scale parameters of an exponential distribution based on singly and doubly censored samples is given When the sample is multiply censored (some middle observations being censored), however, the ML method does not admit explicit solutions In this case we present a simple approximation to the likelihood equation and derive explicit estimators which are linear functions of order statistics Finally, we present some examples to illustrate this method of estimation

Computational aspects of likelihood-based inference for variance components.

Abstract: In this paper, iterative algorithms for computing restricted maximum likelihood (REML) estimates of variance components are discussed in the context of a possibly unbalanced, mixed linear model that contains a single set of m random effects. The coverage includes the Newton-Raphson algorithm, the method of scoring, the EM algorithm, and the method of successive approximations.

Algorithm AS 254: maximum likelihood estimation from grouped and truncated data with finite normal mixture models

Abstract: Description and Purpose Data are often collected in the form of frequencies of observations falling in fixed class intervals. A further feature that is often encountered is truncation in the data; observations below and above certain readings are often not available. Subroutine MGT fits a mixture of a specified number of normal distributions to data collected in this manner. The subroutine can also be used where the data are grouped but not truncated. The fitting procedure uses the EM algorithm (Dempster et al., 1977).

Maximum likelihood estimation using square root information filters

Abstract: The maximum likelihood parameter estimation algorithm is known to provide optimal estimates for linear time-invariant dynamic systems. However, the algorithm is computationally expensive and requires evaluations of the gradient of a log likelihood function and the Fisher information matrix. By using the square-root information filter, a numerically reliable algorithm to compute the required gradient and the Fisher information matrix is developed. The algorithm is a significant improvement over the methods based on the conventional Kalman filter. The square-root information filter relies on the use of orthogonal transformations that are well known for numerical reliability. This algorithm can be extended to real-time system identification and adaptive control. >

Iterative reconstruction-reprojection and the expectation-maximization algorithm

Abstract: The iterative reconstruction-reprojection (IRR) algorithm is a method for estimating missing projections in computed tomography. It is derived as an expectation-maximization (EM) algorithm that increases a suitable likelihood function. The constraint that the data form a consistent set of projections is loosened to require only that the means of the data form a consistent set, thereby suggesting that the algorithm is suitable for use with noisy data. Proofs of convergence to a stationary point and of monotonicity of the sequence of iterates are given. Simulations supporting these results are described. >

Laplace-normal mixtures fitted to wind shear data

Abstract: A mixture model with Laplace and normal components is fitted to wind shear data available in grouped form. A set of equations is presented for iteratively estimating the parameters of the model using an application of the EM algorithm. Twenty-four sets of data are examined with this technique, and the model is found to give a good fit to the data. Some hypotheses about the parameters in the model are discussed in light of the estimates obtained.

A latent time–budget model

Abstract: Time–budgets summarize how the time of objects is distributed over a number of categories. Usually they are collected in object by category matrices with the property that rows of this data matrix add up to one. In this paper we discuss a model for the analysis of time–budgets that used this property. The model approximates the observed time–budgets by weighted sums of a number of latent time–budgets. These latent time–budgets determine the behavior of all objects. Special attention is given to the identification of the model. The model is compared with logcontrast principal component analysis.

A simple EM algorithm for capture-recapture data with categorical covariates.

Abstract: A simple EM algorithm is proposed for obtaining maximum likelihood estimates when fitting a loglinear model to data from k capture-recapture samples with categorical covariates. The method is used to analyze data on screening for the early detection of breast cancer.