Showing papers on "Expectation–maximization algorithm published in 1985"

Maximum likelihood estimation in a class of nonregular cases

Abstract: SUMMARY We consider maximum likelihood estimation of the parameters of a probability density which is zero for x 2, the information matrix is finite and the classical asymptotic properties continue to hold. For cx = 2 the maximum likelihood estimators are asymptotically efficient and normally distributed, but with a different rate of convergence. For 1 < a < 2, the maximum likelihood estimators exist in general, but are not asymptotically normal, while the question of asymptotic efficiency is still unsolved. For cx < 1, the maximum likelihood estimators may not exist at all, but alternatives are proposed. All these results are already known for the case of a single unknown location parameter 0, but are here extended to the case in which there are additional unknown parameters. The paper concludes with a discussion of the applications in extreme value theory.

A Statistical Model for Positron Emission Tomography

Abstract: Positron emission tomography (PET)—still in its research stages—is a technique that promises to open new medical frontiers by enabling physicians to study the metabolic activity of the body in a pictorial manner. Much as in X-ray transmission tomography and other modes of computerized tomography, the quality of the reconstructed image in PET is very sensitive to the mathematical algorithm to be used for reconstruction. In this article, we tailor a mathematical model to the physics of positron emissions, and we use the model to describe the basic image reconstruction problem of PET as a standard problem in statistical estimation from incomplete data. We describe various estimation procedures, such as the maximum likelihood (ML) method (using the EM algorithm), the method of moments, and the least squares method. A computer simulation of a PET experiment is then used to demonstrate the ML and the least squares reconstructions. The main purposes of this article are to report on what we believe is an...

A Constrained Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions

Abstract: The method of maximum likelihood leads to an ill-posed optimization problem in the case of a mixture of normal distributions. Estimation in the univariate case is reformulated using simple constraints into an optimization problem having a strongly consistent, global solution.

Estimation and Hypothesis Testing in Finite Mixture Models

Abstract: SUMMARY Finite mixture models are a useful class of models for application to data. When sample sizes are not large and the number of underlying densities is in question, likelihood ratio tests based on joint maximum likelihood estimation of the mixing parameter, X, and the parameter of the underlying densities, 0, are problematical. Our approach places a prior distribution on X and estimates 0 by maximizing the likelihood of the data given 0 with X integrated out. Advantages of this approach, computational issues using the EM algorithm and directions for further work are discussed. The technique is applied to two examples.

The Use of Sieves to Stabilize Images Produced with the EM Algorithm for Emission Tomography

Abstract: Images produced in emission tomography with the expectation-maximization (EM) algorithm have been observed to become more 'noisy' as the algorithm converges towards the maximum-likelihood estimate. We argue in this paper that there is an instability which is fundamental to maximum-likelihood estimation as it is usually applied and, therefore, is not a result of using the EM algorithm, which is but one numerical implementation for producing maximum-likelihood estimates. We show how Grenader's method of sieves can be used with the EM algorithm to remove the instability and thereby decrease the 'noise' artifact introduced into the images with little or no increase in computational complexity.

Maximum likelihood estimation for mixed continuous and categorical data with missing values

Abstract: SUMMARY Maximum likelihood procedures for analysing mixed continuous and categorical data with missing values are presented. The general location model of Olkin & Tate (1961) and extensions introduced by Krzanowski (1980, 1982) form the basis for our methods. Maximum likelihood estimation with incomplete data is achieved by an application of the EM algorithm (Dempster, Laird & Rubin, 1977). Special cases of the algorithm include Orchard & Woodbury's (1972) algorithm for incomplete normal samples, Fuchs's (1982) algorithms for log linear modelling of partially classified contingency tables, and Day's (1969) algorithm for multivariate normal mixtures. Applications include: (a) imputation of missing values, (b) logistic regression and discriminant analysis with missing predictors and unclassified observations, (c) linear regression with missing continuous and categorical predictors, and (d) parametric cluster analysis with incomplete data. Methods are illustrated using data from the St Louis Risk Research Project. Some key word8: Cluster analysis; Discriminant analysis; EM algorithm; Incomplete data; Linear regression; Logistic regression; Log linear model; Mixture model.

A mixture model for the regression analysis of competing risks data

Abstract: SUMMARY A parametric mixture model provides a regression framework for analysing failure-time data that are subject to censoring and multiple modes of failure. The regression context allows us to adjust for concomitant variables and to assess their effects on the joint distribution of time and type of failure. The mixing parameters correspond to the marginal probabilities of the various failure types and are modelled as logistic functions of the covariates. The hazard rate for each conditional distribution of time to failure, given type of failure, is modelled as the product of a piece-wise exponential function of time and a log-linear function of the covariates. An EM algorithm facilitates the maximum likelihood analysis and illuminates the contributions of the censored observations. The methods are illustrated with data from a heart transplant study and are compared with a cause-specific hazard analysis. The proposed mixture model can also be used to analyse multivariate failure-time data.

Estimation of Latent Group Effects

Abstract: Conventional methods of multivariate normal analysis do not apply when the variables of interest are not observed directly but must be inferred from fallible or incomplete data. A method of estimating such effects by marginal maximum likelihood, implemented by means of an EM algorithm, is proposed. Asymptotic standard errors and likelihood ratio tests of fit are provided. The procedures are illustrated with data from the administration of the Armed Services Vocational Aptitude Battery to a probability sample of American youth.

Log-Linear Models for Doubly Sampled Categorical Data Fitted by the EM Algorithm

Abstract: Double-sampling experiments can be expressed as incomplete multiway contingency tables and analyzed by using techniques appropriate for fitting log-linear models. The framework described can be applied to experiments with any number of factors and measurement devices. Maximum likelihood estimation via the EM algorithm leads to straightforward expressions for covariances of estimates of the parameters and functions of the parameters.

Likelihood Estimation with Normal Mixture Models

Abstract: SUMMARY We consider some of the problems associated with likelihood estimation in the context of a mixture of multivariate normal distributions. Unfortunately with mixture models, the likelihood equation usually has multiple roots and so there is the question of which root to choose. In the case of equal covariance matrices the choice of root is straightforward in the sense that the maximum likelihood estimator exists and is consistent. However, an example is presented to demonstrate that the adoption of a homoscedastic normal model in the presence of some heteroscedasticity can considerably influence the likelihood estimates, in particular of the mixing proportions, and hence the consequent clustering of the sample at hand.

A maximum likelihood approach to frequency-wavenumber analysis

Abstract: This paper is concerned with the likelihood ratio detection and maximum likelihood estimation of plane waves traversing a small array on irregularly placed sensors. A new detection statistic is proposed. This statistic is related to the high resolution procedure of Capon, however it has a known null distribution in the case that no signal is present. Given that a wave is present, the asymptotic distribution of the maximum likelihood estimate of the wave parameters is derived. The problems of detection and estimation are distinguished. They are approached via the fundamental statistical procedures of likelihood ratio testing and maximum likelihood estimation. The solutions of both problems are found to be based upon the same quantity, the maximum likelihood statistic. The development leads to a clarification of the interrelationships and comparative properties of the conventional, the least squares, the high resolution and the new procedure. The approach is via Fourier inference.

Exact maximum likelihood estimation of superimposed exponential signals in noise

Abstract: A unified framework for the exact Maximum Likelihood estimation of the parameters of superimposed exponential signals in noise, encompassing both the single and the multiexperiment cases (respectively the time series and the array problems), is presented. An exact expression for the ML criterion is derived in terms of the prediction polynomial of the noiseless signal, and an iterative algorithm for the maximization of this criterion is presented. A simulation example shows the estimator to be capable of providing more accurate frequency estimates than currently existing techniques.

A Simplified EM Reconstruction Algorithm for TOFPET

Abstract: A simplified expectation-maximization (EM) algorithm for image reconstruction in positron emission tomography with time-of-flight information (TOFPET) has been developed. This new method requires substantially less computation time than the estimated posterior-density weighting (EPDW) method developed by Snyder and Politte [1,2]. Mathematically, the integration of TOFPET images at different angles yields a 2-D image which is the convolution of the true image and a rotationally symmetric point spread function (PSF). The new method uses this PSF to construct the probability functions and employs the summed TOFPET image as the incomplete data set in the expectation step of the EM algorithm; thus, the time-costly angle-by-angle operations required in EPDW are reduced to a single operation, cutting the processing time by a factor of approximately l/M for an image reconstruction using M projection angles. Results from computer simulation studies suggest that this new method may offer image quality superior to that produced by other algorithms.

Optimal multiple source location estimation via the EM algorithm

Abstract: We developed an algorithm for multiple source localization based on the Estimate-Maximize (EM) method. The EM method is an iterative algorithm that converges to the Maximum Likelihood (ML) estimate of the unknown parameters by exploiting the stochastic syctem under consideration. In our case the algorithm will converge to the exact ML estimates of the various sources location parameters, where each iteration increases the likelihood of those parameters.

Deconvolution of Multiple Time Series

Abstract: The problem of estimating simultaneously two convolved vector time series corrupted by additive noise is considered. By regarding one of the series as being stochastic and the other as fixed, it is shown that the fixed component can be estimated by maximizing a frequency domain approximation to the likelihood. The stochastic series is estimated by using an approximation to the conditional mean evaluated at the current maximum likelihood estimators. An example involving a multiple deconvolution of seismic source and receiver functions is given.

Maximum likelihood estimation techniques for concurrent flaw subpopulations

Abstract: Failure of structural materials is often caused by the presence of two or more types of defect subpopulations. The maximum likelihood estimation technique for evaluating the Weibull parameters of these underlying subpopulations in the case of known fracture origin is presented. The maximum likelihood estimation equations are derived, and solved by means of nonlinear programming. The estimators obtained therefrom are tested for both accuracy and consistency against a series of simulation runs. For data sets containing a relatively small sample size, the advantage of the method of maximum likelihood over two established nonparametric techniques is demonstrated.

Information from the maximized likelihood function

Abstract: Suppose x = (x1, ..., xj) is a random sample of either scalar or vector observations from a density f(x, c(), where w( E Q is partitioned into a set 0 = (01, ..., Or) of parameters of direct interest and 4 = (4 1, ..., 4,q) of nuisance parameters. The log likelihood function L(x, w) = I logf(xi, w) is assumed to have continuous first and second derivatives on the parameter space Q and to have a global maximum at an interior point ^) = (a, i) of Q at which the likelihood equations are satisfied. The information in the sample is defined as minus the matrix of second partial derivatives of the log likelihood, that is

Estimation from incomplete data in growth curves models

Abstract: This paper considers a computational method for analysing growth curve data with missing values. We present an algorithm which is often referred as the EM algorithm. The procedure proposed here utilizes the technique of analysis of covariance.

Maximum likelihood estimation of polychoric correlations in r×s ×t contingency tables

Abstract: This paper discusses the maximum likelihood estimation of the polychoric correlation coefficient based on observed frequencies of three polytomous ordinal variables. The underlying latent variables are assumed to have a standardized trivariate normal distribution. The thresholds and correlations are estimated simultaneously via the scoring algorithm. Some practical applications of the method are discussed. An example is reported to illustrate the theory and some technical details are presented in the Appendix.

Maximum Likelihood Regression of Rank-Censored Data

Abstract: Linear regression is a common method for analyzing continuous, cardinal data, but it is inappropriate when the dependent variable is an ordinal ranking The model proposed for analyzing these data sets is the general linear model u = Xβ + e, where the rank of the dependent variable u is observed instead of its value A description is given for a numerical algorithm to evaluate the likelihood function that is efficient enough to permit maximum likelihood estimation of normalized regression coefficients This algorithm can be modified to evaluate the cumulative distribution function of any multivariate normal random vector with nonsingular tridiagonal covariance matrix Large sample properties of the maximum likelihood estimator are provided in the Appendix Finite sample properties of the estimator are examined in a Monte Carlo experiment, and the exact finite sample distribution in one particular case is analyzed The model is applied to voter preference data from a Louis Harris poll

A Preliminary Evaluation of the Use of the EM Algorithm for Estimating Parameters in Dynamic Tracer-Studies

Abstract: The iterative, estimation-maximization algorithm for numerically generating maximum-likelihood estimates of parameters is investigated for use in processing data acquired in dynamic radioactive-tracer studies. This algorithm compared favorably to the best known alternative.

Multiple-Indicator, Multiple-Cause Models for a Single Latent Variable with Ordinal Indicators

Abstract: A MIMIC model is developed that contains a one-dimensional latent variable measured by ordinal indicators. The latent variable can be either discrete and ordered or continuous. Parameters for both the measurement and structural components of the model are estimated simultaneously using the EM algorithm. An example of a sociological application is presented.

Maximum Likelihood Angular Resolution of Multiple Sources

Abstract: normal. If dn denotes the distance of the n The maximum likelihood technique is developed for estimating the directions Of multiple, ClOSelYspaced signal sources from data obtained at the elements of an array. The technique makes use of a certain matrix decomposition which results in some computational simp1 ification. Statistical written as test criteria, which do not require the determination of subjective thresholds, are also applied to the maximum likelihood technique for estimation of the number of sources. The Performance of the maximum likelihood technique and comparison with an eigenvector decomposition technique and the Minimum-Energy technique is determined by computer Simulation. element from some arbitrary first element (origin) and ak(i) denotes the complex amplitude of the ith signal at the origin at time instant tky the observations at the nth element can be

Inferences about latent populations from complex samples

Abstract: Standard procedures for drawing inferences from complex samples do not apply when the variables of interest z are not observed directly, but must be inferred from secondary random variables x that depend on z stochastically. Employing Rubin's (1977) approach to missing data in survey research, we present a procedure by which reasonable inferences can be made in such situations. The key is to represent knowledge about latent variables in the form of a predictive distribution, conditional on manifest variables. It is then possible to obtain the expectations of statistics that would have been computed if the values of the latent variables corresponding to sampled units were known, along with variance estimators that account for uncertainty due to both subject sampling and the latency of z.