Showing papers on "Expectation–maximization algorithm published in 1979"
Posted Content•
TL;DR: In this paper, the joint maximum likelihood estimator of the structural parameters is not consistent as the number of groups increases, with a fixed number of observations per group, and a conditional likelihood function is maximized, conditional on sufficient statistics for the incidental parameters.
Abstract: In data with a group structure, incidental parameters are included to control for missing variables. Applications include longitudinal data and sibling data. In general, the joint maximum likelihood estimator of the structural parameters is not consistent as the number of groups increases, with a fixed number of observations per group. Instead a conditional likelihood function is maximized, conditional on sufficient statistics for the incidental parameters. In the logit case, a standard conditional logit program can be used. Another solution is a random effects model, in which the distribution of the incidental parameters may depend upon the exogenous variables.
2,338 citations
TL;DR: The EM algorithm is shown to provide a slow but sure way of obtaining maximum likelihood estimates of the parameters of interest in compiling a patient record.
Abstract: In compiling a patient record many facets are subject to errors of measurement. A model is presented which allows individual error-rates to be estimated for polytomous facets even when the patient's "true" response is not available. The EM algorithm is shown to provide a slow but sure way of obtaining maximum likelihood estimates of the parameters of interest. Some preliminary experience is reported and the limitations of the method are described.
1,687 citations
01 Jan 1979
TL;DR: The expected distribution of classes in a final classification map can be used to improve classification accuracies by modifying the maximum likelihood decision rule employed in a Bayesian-type classifier to calculate posteriori probabilities of class membership.
405 citations
TL;DR: In this paper, a maximum likelihood estimation procedure of Hawkes' self-exciting point process model is proposed with explicit presentations of the log-likelihood of the model and its gradient and Hessian.
Abstract: A maximum likelihood estimation procedure of Hawkes' self-exciting point process model is proposed with explicit presentations of the log-likelihood of the model and its gradient and Hessian. A simulation method of the process is also presented. Some numerical results are given.
301 citations
168 citations
TL;DR: In this paper, the robustness property of the preliminary test estimator when the assumed restraints may not hold was analyzed for a general multi-sample parametric model and compared with the parallel expressions for the unrestricted and restricted maximum likelihood estimators.
Abstract: Along with the asymptotic distribution, expressions for the asymptotic bias and asymptotic dispersion matrix of the preliminary test maximum likelihood estimator for a general multi-sample parametric model (when the null hypothesis relating to the restraints on the parameters may not hold) are derived and compared with the parallel expressions for the unrestricted and restricted maximum likelihood estimators. This study reveals the robustness property of the preliminary test estimator when the assumed restraints may not hold.
135 citations
TL;DR: In this paper, the authors developed methods of estimation of parameter values and confidence regions by maximum likelihood and Fisher efficient scores starting from Poisson probabilities for the nonlinear spectral functions commonly encountered in X-ray astronomy.
Abstract: Methods of estimation of parameter values and confidence regions by maximum likelihood and Fisher efficient scores starting from Poisson probabilities are developed for the nonlinear spectral functions commonly encountered in X-ray astronomy. It is argued that these methods offer significant advantages over the commonly used alternatives called minimum chi-squared because they rely on less pervasive statistical approximations and so may be expected to remain valid for data of poorer quality. Extensive numerical simulations of the maximum likelihood method are reported which verify that the best-fit parameter value and confidence region calculations are correct over a wide range of input spectra.
125 citations
TL;DR: In this paper, a bias correction using higher derivatives of the log likelihood is shown to be effective in a simulation study of a medical diagnosis problem, where the procedure is illustrated in a medical simulation problem.
Abstract: Maximum likelihood estimates of the parameters for logistic discrimination show considerable bias in small samples, Adjustment by a bias correction using higher derivatives of the log likelihood is shown to be effective in a simulation study. The procedure is illustrated in a medical diagnosis problem.
90 citations
TL;DR: In this article, it was shown that the parameter estimates obtained with the iterative procedure cannot lie outside the allowed interval, and this was later confirmed in a follow-up paper.
Abstract: In this note, we describe the iterative procedure introduced earlier by Goodman to calculate the maximum likelihood estimates of the parameters in latent structure analysis, and we provide here a simple and direct proof of the fact that the parameter estimates obtained with the iterative procedure cannot lie outside the allowed interval. Formann recently stated that Goodman's algorithm can yield parameter estimates that lie outside the allowed interval, and we prove in the present note that Formann's contention is incorrect.
75 citations
TL;DR: In this paper, the authors derived an expression for the exact likelihood function of a stationary process generated by a vector autoregressive-moving average model using concentrated maximum likelihood techniques, and in the process of deriving the likelihood function, a closed form expression for covariance function of the process in terms of the coefficients of the model is derived.
Abstract: SUMMARY By making use of the properties of tensor products, this paper describes the derivation of an expression for the exact likelihood function of a stationary process generated by a vector autoregressive-moving average model using concentrated maximum likelihood techniques. Furthermore, in the process of deriving the likelihood function, a closed form expression for the covariance function of the process in terms of the coefficients of the model is derived.
68 citations
TL;DR: In this article, the strong consistency of the maximum likelihood parameter estimation method for multivariate Gaussian stochastic processes possessing autoregressive moving average (ARMA) representations is established.
Abstract: The strong consistency of the maximum likelihood parameter estimation method is established for multivariate Gaussian stochastic processes possessing autoregressive moving average (ARMA) representations. The demonstration in this paper exploits the ergodic theorem together with results from linear prediction theory.
02 Apr 1979
TL;DR: It can be shown that the likelihood function increases on each iteration of the algorithm, which effectively decouples the uncertainty in the signal and parameter values, thus simplifying the calculation required.
Abstract: For most signal models of interest, Maximum Likelihood (ML) parameter estimation in the presence of noise is a difficult, non-linear problem. A new iterative algorithm has been developed for ML estimation, however, which effectively decouples the uncertainty in the signal and parameter values, thus simplifying the calculation required. It can be shown that the likelihood function increases on each iteration of the algorithm. When applied to a particular pole-zero (ARMA) signal model, each pass consists of a linear smoothing filter followed by solving a set of linear equations for both the pole and zero polynomial coefficients.
TL;DR: In this paper, three methods of estimating the parameters of a power spectrum are analyzed and three methods are shown to give consistent and, under certain conditions, asymptotically equivalent results.
TL;DR: This paper considers the problem of analyzing disease prevalence data from survival experiments in which there may also be some serial sacrifice, and a generalized EM algorithm is utilized to obtain maximum likelihood estimates of the parameters for a broad class of unsaturated models.
Abstract: This paper considers the problem of analyzing disease prevalence data from survival experiments in which there may also be some serial sacrifice. The assumptions needed for "standard" analyses are reviewed in the context of a general model recently proposed by the authors. This model is then reparametrized in log-linear form, and a generalized EM algorithm is utilized to obtain maximum likelihood estimates of the parameters for a broad class of unsaturated models. Tests based on the relative likelihood are proposed to investigate the effects of treatment, time, and the presence of other diseases on the prevalences and lethalities of specific diseases of interest. An example is given, using data from a large experiment to investigate the effects of low-level radiation on laboratory mice. Finally, some possible directions for future research are indicated.
TL;DR: In this article, the authors considered the maximum likelihood estimation of the parameters of a linear structural relationship when repeated observations are available and found that the slope parameter is a root of a fourth-degree polynomial and to be consistent as the number of replicates increases.
Abstract: SUMMARY Maximum likelihood estimation of the parameters of a linear structural relationship when repeated observations are available is considered. The maximum likelihood estimate of the slope parameter is found to be a root of a fourth-degree polynomial and to be consistent as the number of replicates increases. The asymptotic variances and covariances of the estimates of the parameters are obtained through simplified formulae.
TL;DR: In this paper, the authors considered the maximum likelihood estimation of the five parameters of a linear structural relationship y = α + βx when α is known, and the parameters are β, the two variances of observation errors on x and y, the mean and variance of x.
TL;DR: In this article, the authors compare the Newton-Raphson approximation to the maximum likelihood estimate with several other possible procedures, and conclude that it should be used when it yields admissible (positive) results.
Abstract: For some situations the beta-binomial distribution might be used to describe the marginal distribution of test scores for a particular population of examinees. To use this distribution it is necessary to estimate two parameters which characterize the beta-binomial model. One method of estimating these parameters is to approximate the maximum likelihood estimates using some appropriately chosen iterative technique. When the number of examinees is small, however, it is not clear that this method of estimation is justified since the iterative approximations might not converge to the maximum likelihood estimate. Using Monte Carlo techniques, this paper compares the Newton-Raphson approximation to the maximum likelihood estimate with several other possible procedures. It is found that the Newton-Raphson method should be used when it yields admissible (positive) results.
01 Jul 1979
TL;DR: Results of applying CLASSY to real and simulated LANDSAT data are presented and compared with those generated by the iterative self-organizing clustering system algorithm on the same data sets.
Abstract: The CLASSY clustering method alternates maximum likelihood iterative techniques for estimating the parameters of a mixture distribution with an adaptive procedure for splitting, combining, and eliminating the resultant components of the mixture. The adaptive procedure is based on maximizing the fit of a mixture of multivariate normal distributions to the observed data using its first through fourth central moments. It generates estimates of the number of multivariate normal components in the mixture as well as the proportion, mean vector, and covariance matrix for each component. The basic mathematical model for CLASSY and the actual operation of the algorithm as currently implemented are described. Results of applying CLASSY to real and simulated LANDSAT data are presented and compared with those generated by the iterative self-organizing clustering system algorithm on the same data sets.
TL;DR: In this article, an efficient algorithm for computing the W transformation using only about a dozen lines of Fortran or PL/I code is presented. But this algorithm is not suitable for large-scale systems.
Abstract: The W transformation is needed at each step in the maximum likelihood or restricted maximum likelihood procedure for estimation of the parameters of the mixed A.O.V. model. This paper develops an efficient algorithm for computing the W transformation needing only about a dozen lines of Fortran or PL/I code.
TL;DR: In this article, the asymptotic optimality of the restricted maximum likelihood estimates of variance components in the mixed model of analysis of variance was studied and it was shown that such estimates are not only normal but also equivalent to the maximum likelihood in a reasonable sense.
Abstract: In this paper we study the asymptotic optimality of the restricted maximum likelihood estimates of variance components in the mixed model of analysis of variance. Using conceptual design sequences of Miller (1977), under slightly stronger conditions, we show that the restricted maximum likelihood estimates are not only asymptotically normal, but also asymptotically equivalent to the maximum likelihood estimates in a reasonable sense.
Journal Article•
TL;DR: In this paper, a procedure for representing a set of grouped data by a mixture of two normal distributions and two Weibull distributions, using maximum likelihood estimates, is described. But the procedure is not suitable for large sets of data.
Abstract: Computer programs are given for a procedure for representing a set of grouped data by 1). a mixture of two normal distributions, and 2). a mixture of two Weibull distributions, using maximum likelihood estimates...
TL;DR: In this article, a third-order optimum property of the maximum likelihood estimator is extended to not necessarily symmetric loss functions under an appropriate restriction on the class of competing estimators.
TL;DR: In this paper, the authors used the method of scoring to obtain maximum likelihood estimates of the parameters in the White and Clark learning hierarchy validation model, and the hypothesis of inclusion was tested.
Abstract: The method of scoring is used to obtain maximum likelihood estimates of the parameters in the White and Clark learning hierarchy validation model. From the estimate of the proportion of the population possessing only the superordinate skill in a pair of hierarchical skills, and its variance, the hypothesis of inclusion is tested. An illustrative example of the procedure is given.
TL;DR: In this paper, the authors describe maximum likelihood procedures for the Rotterdam model when the model is formulated in relative prices and under the constraints of the theory of rational random behavior, respectively.
TL;DR: In this article, a modification of the Cochrane-Orcutt iterative procedure for the computation of maximum likelihood estimates in a linear regression model with autocorrelated disturbances was proposed, which possesses faster asymptotic convergence rate.
TL;DR: In this article, a frequency-domain algorithm for the maximum likelihood estimation of a dynamic system is developed, where the model of a multivariable linear system is represented by a discrete-type steady-state Kalman filter.
TL;DR: In this article, a simplified two-variable model is generalized to the multivariate case, and the behavior of the likelihood surface is clarified when replications of observations are used to verify the model's correctness.
01 Feb 1979
TL;DR: By combining the best features of various algorithms and taking care to perform each step as efficiently as possible, a decoding scheme was developed which can decode codes which have better performance than those presently in use and yet not require an unreasonable amount of computation.
Abstract: Approximate maximum likelihood decoding algorithms, based upon selecting a small set of candidate code words with the aid of the estimated probability of error of each received symbol, can give performance close to optimum with a reasonable amount of computation. By combining the best features of various algorithms and taking care to perform each step as efficiently as possible, a decoding scheme was developed which can decode codes which have better performance than those presently in use and yet not require an unreasonable amount of computation. The discussion of the details and tradeoffs of presently known efficient optimum and near optimum decoding algorithms leads, naturally, to the one which embodies the best features of all of them.
TL;DR: In this article, the application of the ML-method is demonstrated by estimating parameters in a ball mill-hydrocyclone grinding circuit from flow and density measurements, and the model with the estimated parameters is used to determine the time propagation of mill contents and size distribution.