Showing papers on "Expectation–maximization algorithm published in 2022"

Lightweight Image Super-Resolution With Expectation-Maximization Attention Mechanism

Abstract: In recent years, with the rapid development of deep learning, super-resolution methods based on convolutional neural networks (CNNs) have made great progress. However, the parameters and the required consumption of computing resources of these methods are also increasing to the point that such methods are difficult to implement on devices with low computing power. To address this issue, we propose a lightweight single image super-resolution network with an expectation-maximization attention mechanism (EMASRN) for better balancing performance and applicability. Specifically, a progressive multi-scale feature extraction block (PMSFE) is proposed to extract feature maps of different sizes. Furthermore, we propose an HR-size expectation-maximization attention block (HREMAB) that directly captures the long-range dependencies of HR-size feature maps. We also utilize a feedback network to feed the high-level features of each generation into the next generation’s shallow network. Compared with the existing lightweight single image super-resolution (SISR) methods, our EMASRN reduces the number of parameters by almost one-third. The experimental results demonstrate the superiority of our EMASRN over state-of-the-art lightweight SISR methods in terms of both quantitative metrics and visual quality. The source code can be downloaded at https://github.com/xyzhu1/EMASRN.

Robust Online CSI Estimation in a Complex Environment

Abstract: Channel state information (CSI) estimation is one of the key techniques for improving the performance of wireless communication systems. Meanwhile, the fifth generation wireless communication systems require higher accuracy and lower latency for CSI estimation. In this paper, the methods of noise modeling and online learning are combined to improve the accuracy and reduce the latency. The complex noise environment (considering noise and interference together) is modeled as a specific mixture of Gaussian (MoG) distribution because of its widely approximation capability to any continuous distribution. The MoG CSI estimation (MoG-CE) model and expectation maximization (EM) algorithm are introduced as one of the baseline methods. Further, the parameters of the model can be updated in real time based on the prior knowledge of historical information. Therefore, the online MoG CSI estimation (O-MoG-CE) model and online MoG dynamic CSI estimation (O-MoG-D-CE) model are proposed for time-invariant and time-varying CSI estimations, respectively. The above models can not only self-adapt to various complex communication scenarios robustly but also achieve online and dynamic CSI estimation to improve the accuracy and reduce the latency significantly. In addition, the proposed models can be formulated as standard maximum a posteriori estimations and efficient online expectation maximization (OEM) algorithms are applied for the estimations in a pure machine learning fashion. Comparing with baseline methods, the simulation results demonstrate the superiority of the proposed methods in terms of the accuracy, latency and computation consumption.

Fitting Censored and Truncated Regression Data Using the Mixture of Experts Models

Abstract: The logit-weighted reduced mixture of experts model (LRMoE) is a flexible yet analytically tractable non-linear regression model. Though it has shown usefulness in modeling insurance loss frequencies and severities, model calibration becomes challenging when censored and truncated data are involved, which is common in actuarial practice. In this article, we present an extended expectation–conditional maximization (ECM) algorithm that efficiently fits the LRMoE to random censored and random truncated regression data. The effectiveness of the proposed algorithm is empirically examined through a simulation study. Using real automobile insurance data sets, the usefulness and importance of the proposed algorithm are demonstrated through two actuarial applications: individual claim reserving and deductible ratemaking.

Identification of Nonlinear State-Space Systems With Skewed Measurement Noises

Abstract: In this paper, we consider the identification problem for nonlinear state-space models with skewed measurement noises. The generalized hyperbolic skew Student’s t (GHSkewt) distribution is employed to describe the skewed noises and formulate the hierarchical model of the considered system. A unified framework for estimating unknown states and model parameters is presented based on expectation-maximization (EM) algorithm, in which the forward filtering backward simulation with rejection sampling (RS-FFBSi) is employed to efficiently estimate the smoothing densities of the hidden states, and optimization method is adopted to update model parameters. One numerical study and the electro-mechanical positioning system (EMPS) are employed to verify the effectiveness of the developed approach.

Model‐based clustering of high‐dimensional longitudinal data via regularization

Abstract: We propose a model‐based clustering method for high‐dimensional longitudinal data via regularization in this paper. This study was motivated by the Trial of Activity in Adolescent Girls (TAAG), which aimed to examine multilevel factors related to the change of physical activity by following up a cohort of 783 girls over 10 years from adolescence to early adulthood. Our goal is to identify the intrinsic grouping of subjects with similar patterns of physical activity trajectories and the most relevant predictors within each group. The previous analyses conducted clustering and variable selection in two steps, while our new method can perform the tasks simultaneously. Within each cluster, a linear mixed‐effects model (LMM) is fitted with a doubly penalized likelihood to induce sparsity for parameter estimation and effect selection. The large‐sample joint properties are established, allowing the dimensions of both fixed and random effects to increase at an exponential rate of the sample size, with a general class of penalty functions. Assuming subjects are drawn from a Gaussian mixture distribution, model effects and cluster labels are estimated via a coordinate descent algorithm nested inside the Expectation‐Maximization (EM) algorithm. Bayesian Information Criterion (BIC) is used to determine the optimal number of clusters and the values of tuning parameters. Our numerical studies show that the new method has satisfactory performance and is able to accommodate complex data with multilevel and/or longitudinal effects.

Maximum Likelihood Estimation of Probe Vehicle Penetration Rates and Queue Length Distributions From Probe Vehicle Data

Abstract: Queue length estimation plays an important role in traffic signal control and performance measures of signalized intersections. Traditionally, queue lengths are estimated by applying the shockwave theory to loop detector data. In recent years, the tremendous amount of vehicle trajectory data collected from probe vehicles such as ride-hailing vehicles and connected vehicles provides an alternative approach to queue length estimation. To estimate queue lengths cycle by cycle, many existing methods require the knowledge of the probe vehicle penetration rate and queue length distribution. However, the estimation of the two parameters has not been well studied. This paper proposes a maximum likelihood estimation method that can estimate the parameters from historical probe vehicle data. The maximum likelihood estimation problem is solved by the expectation-maximization (EM) algorithm iteratively. Validation results show that the proposed method could estimate the parameters accurately and thus enable the existing methods to estimate queue lengths cycle by cycle.

The Impact of Total Variation Regularized Expectation Maximization Reconstruction on 68Ga-DOTA-TATE PET/CT Images in Patients With Neuroendocrine Tumor

Abstract: Objective The aim of this study was to investigate the effects of the total variation regularized expectation maximization (TVREM) reconstruction on improving 68Ga-DOTA-TATE PET/CT images compared to the ordered subset expectation maximization (OSEM) reconstruction. Method A total of 17 patients with neuroendocrine tumors who underwent clinical 68Ga-DOTA-TATE PET/CT were involved in this study retrospectively. The PET images were acquired with either 3 min-per-bed (min/bed) acquisition time and reconstructed with OSEM (2 iterations, 20 subsets, and a 3.2-mm Gaussian filter) and TVREM (seven penalization factors = 0.01, 0.07, 0.14, 0.21, 0.28, 0.35, and 0.42) for 2 and 3 min-per-bed (min/bed) acquisition time using list-mode. The SUVmean of the liver, background variability (BV), signal-to-noise ratios (SNR), SUVmax of the lesions and tumor-to-background ratios (TBR) were measured. The mean percentage difference in the SNR and TBR between TVREM with difference penalization factors and OSEM was calculated. Qualitative image quality was evaluated by two experienced radiologists using a 5-point score scale (5-excellent, 1-poor). Results In total, 63 lesions were analyzed in this study. The SUVmean of the liver did not differ significantly between TVREM and OSEM. The BV of all TVREM groups was lower than OSEM groups (all p < 0.05), and the BV of TVREM 2 min/bed group with penalization factor of 0.21 was considered comparable to OSEM 3 min/bed group (p = 0.010 and 0.006). The SNR, SUVmax and TBR were higher for all TVREM groups compared to OSEM groups (all p < 0.05). The mean percentage difference in the SNR and TBR was larger for small lesions (<10 mm) than that for medium (≥10 mm but < 20 mm) and large lesions (≥20 mm). The highest image quality score was given to TVREM 2 min/bed group with penalization factor of 0.21 (3.77 ± 0.26) and TVREM 3 min/bed group with penalization factor of 0.35 (3.77 ± 0.26). Conclusion TVREM could reduce image noise, improve the SNR, SUVmax and TBR of the lesions, and has the potential to preserves the image quality with shorter acquisition time.

Inference and Local Influence Assessment in a Multifactor Skew-Normal Linear Mixed Model

Abstract: This work considers a multifactor linear mixed model under heteroscedasticity in random-effect factors and the skew-normal errors for modeling the correlated datasets. We implement an expectation–maximization (EM) algorithm to achieve the maximum likelihood estimates using conditional distributions of the skew-normal distribution. The EM algorithm is also implemented to extend the local influence approach under three model perturbation schemes in this model. Furthermore, a Monte Carlo simulation is conducted to evaluate the efficiency of the estimators. Finally, a real data set is used to make an illustrative comparison among the following four scenarios: normal/skew-normal errors and heteroscedasticity/homoscedasticity in random-effect factors. The empirical studies show our methodology can improve the estimates when the model errors follow from a skew-normal distribution. In addition, the local influence analysis indicates that our model can decrease the effects of anomalous observations in comparison to normal ones.

Parsimonious hidden Markov models for matrix-variate longitudinal data

Abstract: Hidden Markov models (HMMs) have been extensively used in the univariate and multivariate literature. However, there has been an increased interest in the analysis of matrix-variate data over the recent years. In this manuscript we introduce HMMs for matrix-variate balanced longitudinal data, by assuming a matrix normal distribution in each hidden state. Such data are arranged in a four-way array. To address for possible overparameterization issues, we consider the eigen decomposition of the covariance matrices, leading to a total of 98 HMMs. An expectation-conditional maximization algorithm is discussed for parameter estimation. The proposed models are firstly investigated on simulated data, in terms of parameter recovery, computational times and model selection. Then, they are fitted to a four-way real data set concerning the unemployment rates of the Italian provinces, evaluated by gender and age classes, over the last 16 years.

A model-based clustering of expectation–maximization and K-means algorithms in crime hotspot analysis

Abstract: Hotspot analysis of spatial attributes is a persistent research field in data mining, and applying a model-based clustering procedure is increasingly becoming popular in identifying trends and patterns in datasets on crime events occurring in space. The distributions of potential crime hotspots are parameterized as arising from Gaussian multivariate distributions, whose parameters are estimated by the expectation–maximization (E-M) algorithm, an iterative process with convergence very sensitive to initializations. In this study, a model-based clustering algorithm is explored from the E-M algorithm, initialized by K-means clustering using geodesic distance classification to estimate the model parameters and compared with the classical E-M algorithm, initialized with hierarchical clustering, to identify the distributional patterns of incidence of criminal activities. These model-based clustering algorithms were demonstrated on an open-source large dataset of violent crime activities, which occurred in West Midlands County. Training the data as a Gaussian process, the study identified 12 hotspots of Gaussian mixed models as clusters of an ellipsoidal distribution varying in shape, volume, and orientation, which are mostly found in central parts of boroughs of the study area. The proposed model-based clustering of the E-M algorithm combined with K-means clustering algorithm proved efficient as being fast and stable in convergence with low probability of uncertainty by classifications, producing same classification in some cases when compared to that of the classical E-M and K-means algorithms. The combined model-based clustering techniques applied in the hotspot analysis of criminal activities in space will not only provide insight into crime prediction and resource allocation in combating strategies but also guide researchers to adopt mechanisms for modeling large spatial attributes in data mining.

Robust and Accurate Superquadric Recovery: a Probabilistic Approach

Abstract: Interpreting objects with basic geometric primitives has long been studied in computer vision. Among geometric primitives, superquadrics are well known for their ability to represent a wide range of shapes with few parameters. However, as the first and foremost step, recovering superquadrics accurately and robustly from 3D data still remains challenging. The existing methods are subject to local optima and sensitive to noise and outliers in real-world scenarios, resulting in frequent failure in capturing geometric shapes. In this paper, we propose the first probabilistic method to recover superquadrics from point clouds. Our method builds a Gaussian-uniform mixture model (GUM) on the parametric surface of a superquadric, which explicitly models the generation of outliers and noise. The superquadric recovery is formulated as a Maximum Likelihood Estimation (MLE) problem. We propose an algorithm, Expectation, Maximization, and Switching (EMS), to solve this problem, where: (1) outliers are predicted from the posterior perspective; (2) the superquadric parameter is optimized by the trust-region reflective algorithm; and (3) local optima are avoided by globally searching and switching among parameters encoding similar superquadrics. We show that our method can be extended to the multi-superquadrics recovery for complex objects. The proposed method outperforms the state-of-the-art in terms of accuracy, efficiency, and robustness on both synthetic and real-world datasets. The code is at http://github.com/bmlklwx/EMS-superquadric_fitting.git.

Robust Segmentation Based on Salient Region Detection Coupled Gaussian Mixture Model

Abstract: The impressive progress on image segmentation has been witnessed recently. In this paper, an improved model introducing frequency-tuned salient region detection into Gaussian mixture model (GMM) is proposed, which is named FTGMM. Frequency-tuned salient region detection is added to achieve the saliency map of the original image, which is combined with the original image, and the value of the saliency map is added into the Gaussian mixture model in the form of spatial information weight. The proposed method (FTGMM) calculates the model parameters by the expectation maximization (EM) algorithm with low computational complexity. In the qualitative and quantitative analysis of the experiment, the subjective visual effect and the value of the evaluation index are found to be better than other methods. Therefore, the proposed method (FTGMM) is proven to have high precision and better robustness.

Expectation–Maximization-Based Simultaneous Localization and Mapping for Millimeter-Wave Communication Systems

Abstract: In this paper, we proposed a novel expectation–maximization-based simultaneous localization and mapping (SLAM) algorithm for millimeter-wave (mmW) communication systems. By fully exploiting the geometric relationship among the access point (AP) positions, the angle difference of arrival (ADOA) from the APs and the mobile terminal (MT) position, and regarding the MT positions as the latent variable of the AP positions, the proposed algorithm first reformulates the SLAM problem as the maximum likelihood joint estimation over both the AP positions and the MT positions in a latent variable model. Then, it employs a feasible stochastic approximation expectation–maximization (EM) method to estimate the AP positions. Specifically, the stochastic Monte Carlo approximation is employed to obtain the intractable expectation of the MT positions’ posterior probability in the E-step, and the gradient descent-based optimization is used as a viable substitute for estimating the high-dimensional AP positions in the M-step. Further, it estimates the MT positions and constructs the indoor map based on the estimated AP topology. Due to the efficient processing capability of the stochastic approximation EM method and taking full advantage of the abundant spatial information in the crowd-sourcing ADOA data, the proposed method can achieve a better positioning and mapping performance than the existing geometry-based mmW SLAM method, which usually has to compromise between the computation complexity and the estimation performance. The simulation results confirm the effectiveness of the proposed algorithm.

Multivariate linear mixed models with censored and nonignorable missing outcomes, with application to AIDS studies

Abstract: The analysis of multivariate longitudinal data could encounter some complications due to censorship induced by detection limits of the assay and nonresponse occurring when participants missed scheduled visits intermittently or discontinued participation. This paper establishes a generalization of the multivariate linear mixed model that can accommodate censored responses and nonignorable missing outcomes simultaneously. To account for the nonignorable missingness, the selection approach which decomposes the joint distribution as a marginal distribution for the primary outcome variables and a model describing the missing process conditional on the hypothetical complete data is used. A computationally feasible Monte Carlo expectation conditional maximization algorithm is developed for parameter estimation with the maximum likelihood (ML) method. Furthermore, a general information‐based approach is presented to assess the variability of ML estimators. The techniques for the prediction of censored responses and imputation of missing outcomes are also discussed. The methodology is motivated and exemplified by a real dataset concerning HIV‐AIDS clinical trials. A simulation study is conducted to examine the performance of the proposed method compared with other traditional approaches.

Robust Global Identification of LPV Errors-in-Variables Systems With Incomplete Observations

Abstract: This article develops a robust global strategy for identifying the linear parameter varying (LPV) errors-in-variables (EIVs) systems subjected to randomly missing observations and outliers. The parameter interpolated LPV autoregressive exogenous model with an uncertain/noisy input is investigated and a nonlinear state-space model is considered for the input generation model (IGM). The parameters estimation of the LPV EIV systems with nonideal observations is realized using the expectation–maximization algorithm which is particular effective for the incomplete data issue. To ensure the robustness in the identification, the Student’s t-distribution which is characterized by its adjustable degree of freedom, is used to handle the measurement non-normality. Since the posterior distributions of the latent states in the IGM are also involved in the identification process and they are difficult to calculate directly, the particle filter is introduced to recursively approximate them instead. Finally, the verification examples are given to demonstrate the effectiveness of the developed strategy.

Probabilistic Modeling of Li-Ion Battery Remaining Useful Life

Abstract: The estimation of lithium-ion batteries degradation plays an important role for the correct operation of smart grid and electric vehicle applications. In fact, only healthy battery energy storage systems meet minimum performance in terms of supplied voltage and power. Health prognostic is mandatory to ensure safe and reliable operation of batteries, as unsuccessful operation may cause technical/economic detriments or complete failures. The battery remaining useful life (RUL) depends on several random factors and thus it should be probabilistically characterized to allow the application of decision-making processes. In this article, a hybrid methodology is developed to characterize the RUL even with small amount of data, in view of the typical scarcity of data for adequately fitting the RUL probability distributions. Several distributions are considered in the methodology and compared for such purpose: the inverse Burr (IB) distribution, and mixtures of IB with inverse Gaussian and with inverse Weibull distributions. Expectation-maximization algorithms are specifically developed for the parameter estimation of the considered mixture distributions in the proposed methodology. The IB-based distributions are applied on a RUL dataset created by Monte Carlo sampling on an electrochemical battery model fitted upon given charge/discharge cycles. Numerical experiments are reported to assess the proposal with respect to benchmark distributions.

Machine Learning Based Parameter Estimation of Gaussian Quantum States

Abstract: We propose a machine learning framework for parameter estimation of single mode Gaussian quantum states. Under a Bayesian framework, our approach estimates parameters of suitable prior distributions from measured data. For phase-space displacement and squeezing parameter estimation, this is achieved by introducing Expectation-Maximization (EM) based algorithms, while for phase parameter estimation an empirical Bayes method is applied. The estimated prior distribution parameters along with the observed data are used for finding the optimal Bayesian estimate of the unknown displacement, squeezing, and phase parameters. Our simulation results show that the proposed algorithms have estimation performance that is very close to that of Genie Aided Bayesian estimators, that assume perfect knowledge of the prior parameters. In practical scenarios, when numerical values of the prior distribution parameters are not known beforehand, our proposed methods can be used to find optimal Bayesian estimates from the observed measurement data.

An Approximate Expectation-Maximization for Two-Dimensional Multi-Target Detection

Abstract: We consider the two-dimensional multi-target detection (MTD) problem of estimating a target image from a noisy measurement that contains multiple copies of the image, each randomly rotated and translated. The MTD model serves as a mathematical abstraction of the structure reconstruction problem in single-particle cryo-electron microscopy, the chief motivation of this study. We focus on high noise regimes, where accurate detection of image occurrences within a measurement is impossible. To estimate the image, we develop an expectation-maximization framework that aims to maximize an approximation of the likelihood function. We demonstrate image recovery in highly noisy environments, and show that our framework outperforms the previously studied autocorrelation analysis in a wide range of parameters.

Learning Gaussian Hidden Markov Models From Aggregate Data

Abstract: We consider system identification (learning) problems for Gaussian hidden Markov models (GHMMs). We propose an algorithm to tackle the cases where the data is recorded in aggregate (collective) form generated by a large population of individuals following a certain dynamics. Our parameter learning algorithm is built upon the expectation-maximization algorithm with a novel expectation step proposed recently known as the collective Gaussian forward-backward algorithm. The proposed learning algorithm generalizes the traditional Baum-Welch learning algorithm for GHMMs as it naturally reduces to the latter in case of individual observations.