Expectation–maximization algorithm

About: Expectation–maximization algorithm is a research topic. Over the lifetime, 11823 publications have been published within this topic receiving 528693 citations. The topic is also known as: EM algorithm & Expectation Maximization.

Incomplete Data in Generalized Linear Models

Abstract: This article examines incomplete data for the class of generalized linear models, in which incompleteness is due to partially missing covariates on some observations. Under the assumption that the missing data are missing at random, it is shown that the E step of the EM algorithm for any generalized linear model can be expressed as a weighted complete data log-likelihood when the unobserved covariates are assumed to come from a discrete distribution with finite range. Expressing the E step in this manner allows for a straightforward maximization in the M step, thus leading to maximum likelihood estimates (MLE's) for the parameters. Asymptotic variances of the MLE's are also derived, and results are illustrated with two examples.

Normal/Independent Distributions and Their Applications in Robust Regression

Abstract: Maximum likelihood estimation with nonnormal error distributions provides one method of robust regression. Certain families of normal/independent distributions are particularly attractive for adaptive, robust regression. This article reviews the properties of normal/independent distributions and presents several new results. A major virtue of these distributions is that they lend themselves to EM algorithms for maximum likelihood estimation. EM algorithms are discussed for least Lp regression and for adaptive, robust regression based on the t, slash, and contaminated normal families. Four concrete examples illustrate the performance of the different methods on real data.

A new algorithm for haplotype‐based association analysis: the Stochastic‐EM algorithm

Abstract: Summary It is now widely accepted that haplotypic information can be of great interest for investigating the role of a candidate gene in the etiology of complex diseases. In the absence of family data, haplotypes cannot be deduced from genotypes, except for individuals who are homozygous at all loci or heterozygous at only one site. Statistical methodologies are therefore required for inferring haplotypes from genotypic data and testing their association with a phenotype of interest. Two maximum likelihood algorithms are often used in the context of haplotype-based association studies, the Newton-Raphson (NR) and the Expectation-Maximisation (EM) algorithms. In order to circumvent the limitations of both algorithms, including convergence to local minima and saddle points, we here described how a stochastic version of the EM algorithm, referred to as SEM, could be used for testing haplotypephenotype association. Statistical properties of the SEM algorithm were investigated through a simulation study for a large range of practical situations, including small/large samples and rare/frequent haplotypes, and results were compared to those obtained by use of the standard NR algorithm. Our simulation study indicated that the SEM algorithm provides results similar to those of the NR algorithm, making the SEM algorithm of great interest for haplotype-based association analysis, especially when the number of polymorphisms is quite large.

An experimental comparison of model-based clustering methods

Abstract: We examine methods for clustering in high dimensions. In the first part of the paper, we perform an experimental comparison between three batch clustering algorithms: the Expectation-Maximization (EM) algorithm, a "winner take all" version of the EM algorithm reminiscent of the K-means algorithm, and model-based hierarchical agglomerative clustering. We learn naive-Bayes models with a hidden root node, using high-dimensional discrete-variable data sets (both real and synthetic). We find that the EM algorithm significantly outperforms the other methods, and proceed to investigate the effect of various initialization schemes on the final solution produced by the EM algorithm. The initializations that we consider are (1) parameters sampled from an uninformative prior, (2) random perturbations of the marginal distribution of the data, and (3) the output of hierarchical agglomerative clustering. Although the methods are substantially different, they lead to learned models that are strikingly similar in quality.

Model-Based Expectation-Maximization Source Separation and Localization

Abstract: This paper describes a system, referred to as model-based expectation-maximization source separation and localization (MESSL), for separating and localizing multiple sound sources from an underdetermined reverberant two-channel recording. By clustering individual spectrogram points based on their interaural phase and level differences, MESSL generates masks that can be used to isolate individual sound sources. We first describe a probabilistic model of interaural parameters that can be evaluated at individual spectrogram points. By creating a mixture of these models over sources and delays, the multi-source localization problem is reduced to a collection of single source problems. We derive an expectation-maximization algorithm for computing the maximum-likelihood parameters of this mixture model, and show that these parameters correspond well with interaural parameters measured in isolation. As a byproduct of fitting this mixture model, the algorithm creates probabilistic spectrogram masks that can be used for source separation. In simulated anechoic and reverberant environments, separations using MESSL produced on average a signal-to-distortion ratio 1.6 dB greater and perceptual evaluation of speech quality (PESQ) results 0.27 mean opinion score units greater than four comparable algorithms.