Showing papers on "Expectation–maximization algorithm published in 2018"

Joint Alignment of Multiple Point Sets with Batch and Incremental Expectation-Maximization

Abstract: This paper addresses the problem of registering multiple point sets. Solutions to this problem are often approximated by repeatedly solving for pairwise registration, which results in an uneven treatment of the sets forming a pair: a model set and a data set. The main drawback of this strategy is that the model set may contain noise and outliers, which negatively affects the estimation of the registration parameters. In contrast, the proposed formulation treats all the point sets on an equal footing. Indeed, all the points are drawn from a central Gaussian mixture, hence the registration is cast into a clustering problem. We formally derive batch and incremental EM algorithms that robustly estimate both the GMM parameters and the rotations and translations that optimally align the sets. Moreover, the mixture's means play the role of the registered set of points while the variances provide rich information about the contribution of each component to the alignment. We thoroughly test the proposed algorithms on simulated data and on challenging real data collected with range sensors. We compare them with several state-of-the-art algorithms, and we show their potential for surface reconstruction from depth data.

Blind Channel Estimation for Ambient Backscatter Communication Systems

Abstract: The availability of perfect channel state information is assumed in current ambient-backscatter studies. However, the channel estimation problem for ambient backscatter is radically different from that for traditional wireless systems, where it is common to transmit training (pilot) symbols for this purpose. In this letter, we thus propose a blind channel estimator based on the expectation maximization algorithm to acquire the modulus values of channel parameters. We also obtain the ranges of the initial values of the suggested estimator and derive the modified Bayesian Cramer–Rao bound of the proposed estimator. Finally, simulation results are provided to corroborate our theoretical studies.

Sliced Wasserstein Distance for Learning Gaussian Mixture Models

Abstract: Gaussian mixture models (GMM) are powerful parametric tools with many applications in machine learning and computer vision. Expectation maximization (EM) is the most popular algorithm for estimating the GMM parameters. However, EM guarantees only convergence to a stationary point of the log-likelihood function, which could be arbitrarily worse than the optimal solution. Inspired by the relationship between the negative log-likelihood function and the Kullback-Leibler (KL) divergence, we propose an alternative formulation for estimating the GMM parameters using the sliced Wasserstein distance, which gives rise to a new algorithm. Specifically, we propose minimizing the sliced-Wasserstein distance between the mixture model and the data distribution with respect to the GMM parameters. In contrast to the KL-divergence, the energy landscape for the sliced-Wasserstein distance is more well-behaved and therefore more suitable for a stochastic gradient descent scheme to obtain the optimal GMM parameters. We show that our formulation results in parameter estimates that are more robust to random initializations and demonstrate that it can estimate high-dimensional data distributions more faithfully than the EM algorithm.

A fast likelihood solution to the genetic clustering problem

Abstract: The investigation of genetic clusters in natural populations is an ubiquitous problem in a range of fields relying on the analysis of genetic data, such as molecular ecology, conservation biology and microbiology. Typically, genetic clusters are defined as distinct panmictic populations, or parental groups in the context of hybridisation. Two types of methods have been developed for identifying such clusters: model-based methods, which are usually computer-intensive but yield results which can be interpreted in the light of an explicit population genetic model, and geometric approaches, which are less interpretable but remarkably faster.Here, we introduce snapclust, a fast maximum-likelihood solution to the genetic clustering problem, which allies the advantages of both model-based and geometric approaches. Our method relies on maximising the likelihood of a fixed number of panmictic populations, using a combination of geometric approach and fast likelihood optimisation, using the Expectation-Maximisation (EM) algorithm. It can be used for assigning genotypes to populations and optionally identify various types of hybrids between two parental populations. Several goodness-of-fit statistics can also be used to guide the choice of the retained number of clusters.Using extensive simulations, we show that snapclust performs comparably to current gold standards for genetic clustering as well as hybrid detection, with some advantages for identifying hybrids after several backcrosses, while being orders of magnitude faster than other model-based methods. We also illustrate how snapclust can be used for identifying the optimal number of clusters, and subsequently assign individuals to various hybrid classes simulated from an empirical microsatellite dataset. snapclust is implemented in the package adegenet for the free software R, and is therefore easily integrated into existing pipelines for genetic data analysis. It can be applied to any kind of co-dominant markers, and can easily be extended to more complex models including, for instance, varying ploidy levels. Given its flexibility and computer-efficiency, it provides a useful complement to the existing toolbox for the study of genetic diversity in natural populations.

A Gaussian Mixture Model Representation of Endmember Variability in Hyperspectral Unmixing.

Abstract: Hyperspectral unmixing while considering endmember variability is usually performed by the normal compositional model, where the endmembers for each pixel are assumed to be sampled from unimodal Gaussian distributions. However, in real applications, the distribution of a material is often not Gaussian. In this paper, we use Gaussian mixture models (GMM) to represent endmember variability. We show, given the GMM starting premise, that the distribution of the mixed pixel (under the linear mixing model) is also a GMM (and this is shown from two perspectives). The first perspective originates from random variable transformations and gives a conditional density function of the pixels given the abundances and GMM parameters. With proper smoothness and sparsity prior constraints on the abundances, the conditional density function leads to a standard maximum a posteriori (MAP) problem which can be solved using generalized expectation maximization. The second perspective originates from marginalizing over the endmembers in the GMM, which provides us with a foundation to solve for the endmembers at each pixel. Hence, compared to the other distribution based methods, our model can not only estimate the abundances and distribution parameters, but also the distinct endmember set for each pixel. We tested the proposed GMM on several synthetic and real datasets, and showed its potential by comparing it to current popular methods.

Likelihood-Based Automatic Modulation Classification in OFDM With Index Modulation

Abstract: In orthogonal frequency division multiplexing (OFDM) with index modulation, the modulation parameters to be classified include both the signal constellation and the number of active subcarriers. This is different from conventional OFDM schemes where only the signal constellation needs to be classified. In this paper, to solve this challenging problem, the likelihood-based automatic modulation classification (AMC) is studied. First, in the scenario with known channel state information (CSI), two classifiers based on average likelihood ratio test (ALRT) and hybrid likelihood ratio test (HLRT), respectively, are derived. Concretely, in HLRT-based classifier, energy-based detector, and log-likelihood ratio-based detector are employed to identify the active subcarriers. Second, a HLRT-based blind AMC is proposed in the scenario with unknown CSI, where the efficient implementation of expectation maximization algorithm is presented to estimate channel fading coefficients and noise variance. Finally, the effectiveness of the proposed AMC algorithms is confirmed by computer simulations.

Blind Estimation of Sparse Broadband Massive MIMO Channels With Ideal and One-bit ADCs

Abstract: We study the maximum likelihood problem for the blind estimation of massive mmWave MIMO channels while taking into account their underlying sparse structure, the temporal shifts across antennas in the broadband regime, and ultimately one-bit quantization at the receiver. The sparsity in the angular domain is exploited as a key property to enable the unambiguous blind separation between user's channels. The main advantage of this approach is the fact that the overhead due to pilot sequences can be dramatically reduced especially when operating at low SNR per antenna. In addition, as sparsity is the only assumption made about the channel, the proposed method is robust with respect to the statistical properties of the channel and data and allows the channel estimation and the separation of interfering users from adjacent base stations to be performed in rapidly time-varying scenarios. For the case of one-bit receivers, a blind channel estimation is proposed that relies on the expectation maximization algorithm. Additionally, performance limits are derived based on the Clairvoyant Cramer–Rao lower bound. Simulation results demonstrate that this maximum likelihood formulation yields superior estimation accuracy in the narrowband as well as the wideband regime with reasonable computational complexity and limited model assumptions.

Theory and Experiments on Vector Quantized Autoencoders

Abstract: Deep neural networks with discrete latent variables offer the promise of better symbolic reasoning, and learning abstractions that are more useful to new tasks. There has been a surge in interest in discrete latent variable models, however, despite several recent improvements, the training of discrete latent variable models has remained challenging and their performance has mostly failed to match their continuous counterparts. Recent work on vector quantized autoencoders (VQ-VAE) has made substantial progress in this direction, with its perplexity almost matching that of a VAE on datasets such as CIFAR-10. In this work, we investigate an alternate training technique for VQ-VAE, inspired by its connection to the Expectation Maximization (EM) algorithm. Training the discrete bottleneck with EM helps us achieve better image generation results on CIFAR-10, and together with knowledge distillation, allows us to develop a non-autoregressive machine translation model whose accuracy almost matches a strong greedy autoregressive baseline Transformer, while being 3.3 times faster at inference.

A Sampling-Based Bayesian Approach for Cooperative Multiagent Online Search With Resource Constraints

Abstract: This paper presents a cooperative multiagent search algorithm to solve the problem of searching for a target on a 2-D plane under multiple constraints. A Bayesian framework is used to update the local probability density functions (PDFs) of the target when the agents obtain observation information. To obtain the global PDF used for decision making, a sampling-based logarithmic opinion pool algorithm is proposed to fuse the local PDFs, and a particle sampling approach is used to represent the continuous PDF. Then the Gaussian mixture model (GMM) is applied to reconstitute the global PDF from the particles, and a weighted expectation maximization algorithm is presented to estimate the parameters of the GMM. Furthermore, we propose an optimization objective which aims to guide agents to find the target with less resource consumptions, and to keep the resource consumption of each agent balanced simultaneously. To this end, a utility function-based optimization problem is put forward, and it is solved by a gradient-based approach. Several contrastive simulations demonstrate that compared with other existing approaches, the proposed one uses less overall resources and shows a better performance of balancing the resource consumption.

Two-Step Estimation of Models Between Latent Classes and External Variables

Abstract: We consider models which combine latent class measurement models for categorical latent variables with structural regression models for the relationships between the latent classes and observed explanatory and response variables. We propose a two-step method of estimating such models. In its first step, the measurement model is estimated alone, and in the second step the parameters of this measurement model are held fixed when the structural model is estimated. Simulation studies and applied examples suggest that the two-step method is an attractive alternative to existing one-step and three-step methods. We derive estimated standard errors for the two-step estimates of the structural model which account for the uncertainty from both steps of the estimation, and show how the method can be implemented in existing software for latent variable modelling.

A semiparametric extension of the stochastic block model for longitudinal networks

Abstract: We propose an extension of the stochastic block model for recurrent interaction events in continuous time, where every individual belongs to a latent group and conditional interactions between two individuals follow an inhomogeneous Poisson process with intensity driven by the individuals’ latent groups. We show that the model is identifiable and estimate it with a semiparametric variational expectation-maximization algorithm. We develop two versions of the method, one using a nonparametric histogram approach with an adaptive choice of the partition size, and the other using kernel intensity estimators. We select the number of latent groups by an integrated classification likelihood criterion. We demonstrate the performance of our procedure on synthetic experiments, analyse two datasets to illustrate the utility of our approach, and comment on competing methods.

Efficient Estimation of Component Interactions for Cascading Failure Analysis by EM Algorithm

Abstract: Due to the lack of information about the causes of outages, the estimation of the interactions between component failures that capture the general outage propagation patterns is a typical parameter estimation problem with incomplete data. In this paper, we estimate these interactions by the expectation maximization (EM) algorithm. The proposed method is validated with simulated cascading failure data from the AC OPA model on the IEEE 118-bus system. The EM algorithm can accurately estimate the interactions and identify the key links and key components using only a small number of the original cascades from a detailed cascading blackout model, which is critical for online cascading failure analysis and decision making. Compared with AC OPA simulation, the highly probabilistic interaction model simulation based on the proposed interaction estimation method can achieve a speed-up of 100.61.

Narrowband and Wideband Off-Grid Direction-of-Arrival Estimation via Sparse Bayesian Learning

Abstract: The sparse Bayesian learning based relevance vector machine (SBLRVM) algorithm is a promising algorithm to estimate the directions-of-arrival (DOAs) of multiple narrowband signals. The parameters involved in the DOA estimation model are automatically estimated by the algorithm that makes it more attractive than the deterministic sparsity based DOA estimation algorithms in which fine-tuning of parameters is necessary. However, one limitation of the algorithm is that it assumes the DOAs of the signals to be exactly aligned with the angular grids, which may not be true in practice. In this paper, we first propose an off-grid version of the narrowband SBLRVM algorithm. Next, we propose an off-grid wideband SBLRVM algorithm. The algorithms assume that the true scenario DOAs of the signals are not exactly aligned with the angular grids and the parameters of the algorithms are automatically estimated by the expectation maximization approach. In the wideband DOA estimation algorithm, we estimate one spatial power spectrum by simultaneously exploiting sparsity from all frequency bins. We demonstrate the application of the proposed algorithms by analyzing data from the shallow water HF $\mathbf {97}$ ocean acoustic experiment. The estimated DOAs of a narrowband tonal from the experiment by using our proposed narrowband DOA estimation algorithm are consistent with the nonadaptive conventional beamformer. Processing a wideband chirp from the experiment shows that estimating one spatial power spectrum by simultaneously exploiting sparsity from all frequency bins using the proposed wideband DOA estimation algorithm is a more valuable processor than an incoherent combination of the power spectra from the individual frequency bins estimated using the proposed narrowband DOA estimation algorithm. Moreover, since our proposed algorithms are off-grid algorithms, an empirical analysis for the choice of the discretization interval of the angular spread is not required as opposed to the on-grid DOA estimation algorithms. This results in a reduced computational complexity.

Semi-blind Channel Estimation for Multiuser Massive MIMO Systems

Abstract: Motivated by recent developments in time-division duplex massive multiple-input multiple-output (MIMO) systems, this paper investigates semi-blind channel estimation for multiuser MIMO systems. An expectation-maximization (EM) algorithm is derived for semi-blind channel estimation and a tractable EM algorithm is obtained by assuming a Gaussian distribution for the unknown data symbols, which improves channel estimates even when the data symbols are drawn from a finite constellation, such as quadrature phase-shift keying. An alternate EM algorithm is also derived by employing suitable priors on the channel coefficients and it is shown to outperform the EM algorithm with no priors in the low signal-to-noise ratio regime. To analytically understand the performance of the semi-blind scheme, Cramer–Rao bounds (CRBs) for semi-blind channel estimation are derived for deterministic and stochastic (Gaussian) data symbol models. To get insight into the behavior of a massive MIMO system, the asymptotic behavior of the CRBs as the number of antennas at the base station grows is analyzed. The numerical results show the benefits of semi-blind estimation algorithm as measured by the mean squared error. In particular, the performance of the EM algorithm becomes closer to the genie-aided maximum likelihood estimator based on known data symbols as the number of antennas increases. This result is consistent with the asymptotic analysis of the two CRBs indicating that semi-blind channel estimation for massive MIMO systems is very promising.

Degradation in Common Dynamic Environments

Abstract: Degradation studies are often used to assess reliability of products subject to degradation-induced soft failures. Because of limited test resources, several test subjects may have to share a test rig and have their degradation measured by the same operator. The common environments experienced by subjects in the same group introduce significant interindividual correlations in their degradation, which is known as the block effect. In the present article, the Wiener process is used to model product degradation, and the group-specific random environments are captured using a stochastic time scale. Both semiparametric and parametric estimation procedures are developed for the model. Maximum likelihood estimations of the model parameters for both the semiparametric and parametric models are obtained with the help of the EM algorithm. Performance of the maximum likelihood estimators is validated through large sample asymptotics and small sample simulations. The proposed models are illustrated by an appl...

Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods

Abstract: Fil: Pulido, Manuel Arturo Consejo Nacional de Investigaciones Cientificas y Tecnicas Centro Cientifico Tecnologico Conicet - Nordeste Instituto de Modelado e Innovacion Tecnologica Universidad Nacional del Nordeste Facultad de Ciencias Exactas Naturales y Agrimensura Instituto de Modelado e Innovacion Tecnologica; Argentina

Estimating the parameters of an inverse Weibull distribution under progressive type-I interval censoring

Abstract: In this paper, we consider the problem of estimating unknown parameters of an inverse Weibull distribution when it is known that samples are progressive type-I interval censored. We propose an EM algorithm to obtain maximum likelihood estimates and mid point estimates. For comparison purpose Bayes estimates are also obtained under the square error loss function. A simulation study is conducted to access the performance of the proposed estimators and recommendations are made on the basis of simulation results. A real data set is also analyzed in detail for an illustration purpose. Finally, by making use of expected Fisher information matrix various inspection times and optimal censoring schemes are obtained.

Finding the Number of Normal Groups in Model-Based Clustering via Constrained Likelihoods

Abstract: Deciding the number of clusters k is one of the most difficult problems in cluster analysis. For this purpose, complexity-penalized likelihood approaches have been introduced in model-based cluster...

Approximate Bayes learning of stochastic differential equations.

Abstract: We introduce a nonparametric approach for estimating drift and diffusion functions in systems of stochastic differential equations from observations of the state vector. Gaussian processes are used as flexible models for these functions, and estimates are calculated directly from dense data sets using Gaussian process regression. We develop an approximate expectation maximization algorithm to deal with the unobserved, latent dynamics between sparse observations. The posterior over states is approximated by a piecewise linearized process of the Ornstein-Uhlenbeck type and the maximum a posteriori estimation of the drift is facilitated by a sparse Gaussian process approximation.

Technical Note—An Expectation-Maximization Algorithm to Estimate the Parameters of the Markov Chain Choice Model

Abstract: We develop an expectation-maximization algorithm to estimate the parameters of the Markov chain choice model. In this choice model, a customer arrives into the system to purchase a certain product....

logbin: An R Package for Relative Risk Regression Using the Log-Binomial Model

Abstract: Relative risk regression using a log-link binomial generalized linear model (GLM) is an important tool for the analysis of binary outcomes. However, Fisher scoring, which is the standard method for fitting GLMs in statistical software, may have difficulties in converging to the maximum likelihood estimate due to implicit parameter constraints. logbin is an R package that implements several algorithms for fitting relative risk regression models, allowing stable maximum likelihood estimation while ensuring the required parameter constraints are obeyed. We describe the logbin package and examine its stability and speed for different computational algorithms. We also describe how the package may be used to include flexible semi-parametric terms in relative risk regression models.

Parameter Estimation of a Gaussian Mixture Model for Wind Power Forecast Error by Riemann L-BFGS Optimization

Abstract: With the increasing penetration of wind power into the electricity grid, wind power forecast error analysis plays an important role in operations scheduling To better describe the characteristics of power forecast error, a probability density function should be established Compared with the Kernel density estimation method, this paper adopts the Gaussian mixture model (GMM), which is flexible enough to capture different error distribution characteristics, such as bias, heavy tail, multi-peak, and so on In addition, for GMM parameter estimation, when dealing with a large number of multi-dimensional data sets or unbalanced overlapping mixtures, the expectation maximization (EM) algorithm shows a slower convergence speed and requires a high number of iterations In this paper, a new L-BFGS optimization method, based on the Riemannian manifold, is used for GMM parameter estimation Based on actual wind power forecast error data, the suitability of the model and the new optimization algorithm was verified in large, multi-dimensional data sets The new optimization algorithm has fewer iterations than the EM algorithm, with an improved convergence speed