Expectation–maximization algorithm

About: Expectation–maximization algorithm is a research topic. Over the lifetime, 11823 publications have been published within this topic receiving 528693 citations. The topic is also known as: EM algorithm & Expectation Maximization.

Weighted likelihood, pseudo-likelihood and maximum likelihood methods for logistic regression analysis of two-stage data.

Abstract: General approaches to the fitting of binary response models to data collected in two-stage and other stratified sampling designs include weighted likelihood, pseudo-likelihood and full maximum likelihood. In previous work the authors developed the large sample theory and methodology for fitting of logistic regression models to two-stage case-control data using full maximum likelihood. The present paper describes computational algorithms that permit efficient estimation of regression coefficients using weighted, pseudo- and full maximum likelihood. It also presents results of a simulation study involving continuous covariables where maximum likelihood clearly outperformed the other two methods and discusses the analysis of data from three bona fide case-control studies that illustrate some important relationships among the three methods. A concluding section discusses the application of two-stage methods to case-control studies with validation subsampling for control of measurement error.

Models for Categorical Data with Nonignorable Nonresponse

Abstract: When categorical outcomes are subject to nonignorable nonresponse, log-linear models may be used to adjust for the nonresponse. The models are fitted to the data in an augmented frequency table in which one index corresponds to whether or not the subject is a respondent. The likelihood function is maximized over pseudo-observed cell frequencies with respect to this log-linear model using an EM algorithm. Each E step of the EM algorithm determines the pseudo-observed cell frequencies, and the M step yields the maximum likelihood estimators (MLE's) of these pseudo-observed cell frequencies. This approach may produce boundary estimates for the expected cell frequencies of the nonrespondents. In these cases the estimators of the log-linear model parameters are not uniquely determined and may be unstable. Following the approach of Clogg et al., we propose a Bayesian method that uses smoothing constants to adjust the pseudo-observed cell frequencies so that the solution is not on the boundary. The role...

Improved hidden Markov models in the wavelet-domain

Abstract: Wavelet-domain hidden Markov models (HMMs), in particular the hidden Markov tree (HMT) model, have been introduced and applied to signal and image processing, e.g., signal denoising. We develop a simple initialization scheme for the efficient HMT model training and then propose a new four-state HMT model called HMT-2. We find that the new initialization scheme fits the HMT-2 model well. Experimental results show that the performance of signal denoising using the HMT-2 model is often improved over the two-state HMT model developed by Crouse et al. (see ibid., vol.46, p.886-902, 1998).

Rayleigh-Rice Mixture Parameter Estimation via EM Algorithm for Change Detection in Multispectral Images

Abstract: The problem of estimating the parameters of a Rayleigh-Rice mixture density is often encountered in image analysis (e.g., remote sensing and medical image processing). In this paper, we address this general problem in the framework of change detection (CD) in multitemporal and multispectral images. One widely used approach to CD in multispectral images is based on the change vector analysis. Here, the distribution of the magnitude of the difference image can be theoretically modeled by a Rayleigh-Rice mixture density. However, given the complexity of this model, in applications, a Gaussian-mixture approximation is often considered, which may affect the CD results. In this paper, we present a novel technique for parameter estimation of the Rayleigh-Rice density that is based on a specific definition of the expectation-maximization algorithm. The proposed technique, which is characterized by good theoretical properties, iteratively updates the parameters and does not depend on specific optimization routines. Several numerical experiments on synthetic data demonstrate the effectiveness of the method, which is general and can be applied to any image processing problem involving the Rayleigh-Rice mixture density. In the CD context, the Rayleigh-Rice model (which is theoretically derived) outperforms other empirical models. Experiments on real multitemporal and multispectral remote sensing images confirm the validity of the model by returning significantly higher CD accuracies than those obtained by using the state-of-the-art approaches.

LTL model checking of interval markov chains

Abstract: Interval Markov chains (IMCs) generalize ordinary Markov chains by having interval-valued transition probabilities. They are useful for modeling systems in which some transition probabilities depend on an unknown environment, are only approximately known, or are parameters that can be controlled. We consider the problem of computing values for the unknown probabilities in an IMC that maximize the probability of satisfying an ω-regular specification. We give new upper and lower bounds on the complexity of this problem. We then describe an approach based on an expectation maximization algorithm. We provide some analytical guarantees on the algorithm, and show how it can be combined with translation of logic to automata. We give experiments showing that the resulting system gives a practical approach to model checking IMCs.