Showing papers on "Expectation–maximization algorithm published in 2015"

An Adaptive Prognostic Approach via Nonlinear Degradation Modeling: Application to Battery Data

Abstract: Remaining useful life (RUL) estimation via degradation modeling is considered as one of the most central components in prognostics and health management. Current RUL estimation studies mainly focus on linear stochastic models, and the results under nonlinear models are relatively limited in literature. Even in nonlinear degradation modeling, the estimated RUL is aimed at a population of systems of the same type or depend only on the current degradation observation. In this paper, an adaptive and nonlinear prognostic model is presented to estimate RUL using a system's history of the observed data to date. Specifically, a general nonlinear stochastic process with a time-dependent drift coefficient is first adopted to characterize the dynamics and nonlinearity of the degradation process. In order to render the RUL estimation depending on the degradation history to date, a state-space model is constructed, and Kalman filtering is applied to update one key parameter in the drifting function through treating this parameter as an unobserved state variable. To update the hidden state and other parameters in the state-space model simultaneously and recursively, the expectation maximization algorithm is used in conjunction with Kalman smoother to achieve this aim. The probability density function of the estimated RUL is derived with an explicit form, and some commonly used results under linear models turn out to be its special cases. Finally, the implementation of the presented approach is illustrated by numerical simulations, and an application for estimating the RUL of lithium-ion batteries is used to demonstrate the superiority of the method.

Statistical clustering of temporal networks through a dynamic stochastic block model

Abstract: Statistical node clustering in discrete time dynamic networks is an emerging field that raises many challenges. Here, we explore statistical properties and frequentist inference in a model that combines a stochastic block model (SBM) for its static part with independent Markov chains for the evolution of the nodes groups through time. We model binary data as well as weighted dynamic random graphs (with discrete or continuous edges values). Our approach, motivated by the importance of controlling for label switching issues across the different time steps, focuses on detecting groups characterized by a stable within group connectivity behavior. We study identifiability of the model parameters, propose an inference procedure based on a variational expectation maximization algorithm as well as a model selection criterion to select for the number of groups. We carefully discuss our initialization strategy which plays an important role in the method and compare our procedure with existing ones on synthetic datasets. We also illustrate our approach on dynamic contact networks, one of encounters among high school students and two others on animal interactions. An implementation of the method is available as a R package called dynsbm.

Inverse Gaussian Processes With Random Effects and Explanatory Variables for Degradation Data

Abstract: Degradation models are widely used to assess the lifetime information of highly reliable products. This study proposes a degradation model based on an inverse normal-gamma mixture of an inverse Gaussian process. This article presents the properties of the lifetime distribution and parameter estimation using the EM-type algorithm, in addition to providing a simple model-checking procedure to assess the validity of different stochastic processes. Several case applications are performed to demonstrate the advantages of the proposed model with random effects and explanatory variables. Technical details, data, and R code are available online as supplementary materials.

Probabilistic slow feature analysis‐based representation learning from massive process data for soft sensor modeling

Abstract: Latent variable (LV) models provide explicit representations of underlying driving forces of process variations and retain the dominant information of process data In this study, slow features as temporally correlated LVs are derived using probabilistic slow feature analysis Slow features evolving in a state-space form effectively represent nominal variations of processes, some of which are potentially correlated to quality variables and hence help improving the prediction performance of soft sensors An efficient EM algorithm is proposed to estimate parameters of the probabilistic model, which turns out to be suitable for analyzing massive process data Two criteria are ∗To whom correspondence should be addressed †Tsinghua University ‡University of Alberta 1 also proposed to select quality-relevant slow features The validity and advantages of the proposed method are demonstrated via two case studies

Estimation and selection for the latent block model on categorical data

Abstract: This paper deals with estimation and model selection in the Latent Block Model (LBM) for categorical data. First, after providing sufficient conditions ensuring the identifiability of this model, we generalise estimation procedures and model selection criteria derived for binary data. Secondly, we develop Bayesian inference through Gibbs sampling and with a well calibrated non informative prior distribution, in order to get the MAP estimator: this is proved to avoid the traps encountered by the LBM with the maximum likelihood methodology. Then model selection criteria are presented. In particular an exact expression of the integrated completed likelihood criterion requiring no asymptotic approximation is derived. Finally numerical experiments on both simulated and real data sets highlight the appeal of the proposed estimation and model selection procedures.

Additive regularization of topic models

Abstract: Probabilistic topic modeling of text collections has been recently developed mainly within the framework of graphical models and Bayesian inference. In this paper we introduce an alternative semi-probabilistic approach, which we call additive regularization of topic models (ARTM). Instead of building a purely probabilistic generative model of text we regularize an ill-posed problem of stochastic matrix factorization by maximizing a weighted sum of the log-likelihood and additional criteria. This approach enables us to combine probabilistic assumptions with linguistic and problem-specific requirements in a single multi-objective topic model. In the theoretical part of the work we derive the regularized EM-algorithm and provide a pool of regularizers, which can be applied together in any combination. We show that many models previously developed within Bayesian framework can be inferred easier within ARTM and in some cases generalized. In the experimental part we show that a combination of sparsing, smoothing, and decorrelation improves several quality measures at once with almost no loss of the likelihood.

Dynamically Modeling Patient's Health State from Electronic Medical Records: A Time Series Approach

Abstract: In this paper, we present a method to dynamically estimate the probability of mortality inside the Intensive Care Unit (ICU) by combining heterogeneous data. We propose a method based on Generalized Linear Dynamic Models that models the probability of mortality as a latent state that evolves over time. This framework allows us to combine different types of features (lab results, vital signs readings, doctor and nurse notes, etc) into a single state, which is updated each time new patient data is observed. In addition, we include the use of text features, based on medical noun phrase extraction and Statistical Topic Models. These features provide context about the patient that cannot be captured when only numerical features are used. We fill out the missing values using a Regularized Expectation Maximization based method assuming temporal data. We test our proposed approach using 15,000 Electronic Medical Records (EMRs) obtained from the MIMIC II public dataset. Experimental results show that the proposed model allows us to detect an increase in the probability of mortality before it occurs. We report an AUC 0.8657. Our proposed model clearly outperforms other methods of the literature in terms of sensitivity with 0.7885 compared to 0.6559 of Naive Bayes and F-score with 0.5929 compared to 0.4662 of Apache III score after 24 hours.

Iterative Ensemble Smoother as an Approximate Solution to a Regularized Minimum-Average-Cost Problem: Theory and Applications

Abstract: The focus of this work is on an alternative implementation of the iterative ensemble smoother (iES). We show that iteration formulae similar to those used in [3,6] can be derived by adopting a regularized Levenberg-Marquardt (RLM) algorithm [14] to approximately solve a minimum-average-cost (MAC) problem. This not only leads to an alternative theoretical tool in understanding and analyzing the behaviour of the aforementioned iES, but also provides insights and guidelines for further developments of the smoothing algorithms. For illustration, we compare the performance of an implementation of the RLM-MAC algorithm to that of the approximate iES used in [3] in three numerical examples: an initial condition estimation problem in a strongly nonlinear system, a facies estimation problem in a 2D reservoir and the history matching problem in the Brugge field case. In these three specific cases, the RLM-MAC algorithm exhibits comparable or even better performance, especially in the strongly nonlinear system. Introduction For data assimilation problems there are different ways in utilizing the available observations. While certain data assimilation algorithms, for instance, the ensemble Kalman filter (EnKF, see, for example, [2,8]), assimilate the observations sequentially in time, other data assimilation algorithms may instead Xiaodong Luo · Andreas S. Stordal · Rolf J. Lorentzen · Geir Naevdal International Research Institute of Stavanger (IRIS), 5008 Bergen, Norway E-mail: xiaodong.luo@iris.no 2 Xiaodong Luo et al. collect the observations at different time instants and assimilate them simultaneously. Examples in this aspect include the ensemble smoother (ES, see, for example, [9]) and its iterative variants. The EnKF has been widely used for reservoir data assimilation (history matching) problems since its introduction to the community of petroleum engineering [26]. The applications of the ES to reservoir data assimilation problems are also investigated recently (see, for example, [33]). Compared to the EnKF, the ES has certain technical advantages, including, for instance, avoiding the restarts associated with each update step in the EnKF for certain reservoir simulators (e.g., ECLIPSE c © [1]) and also having fewer variables to update. The formal benefit (avoiding restarts) may result in a significant reduction of simulation time in certain circumstances [33], while the latter (having fewer variables) can reduce the amount of computer memory in use. To further improve the performance of the ES, some iterative ensemble smoothers (iES) are suggested in the literature, in which the iterations are carried out in the forms of certain iterative optimization algorithms, e.g., the Gaussian-Newton [4,10,16] or the Levenberg-Marquardt method [3,6,13,18,21,22], or in the context of adaptive Gaussian mixture (AGM, see [34]). In [13,22], the iteration formulae are adopted following the regularized Levenberg-Marquardt (RLM) algorithm in the deterministic inverse problem theory (see, for example, [7,15]). Essentially these formulae aim to find a single solution of the inverse problem, and the gradient involved in the iteration is obtained either through the adjoint model [13], or through a stochastic approximation method [22]. While in [6], the iteration formula is derived based on the idea that, the final result of the iES should be equal to the estimate of the EnKF, at least for linear systems with Gaussian model and observation errors. Consequently, this algorithm is called ensemble smoother with multiple data assimilation (ES-MDA for short). On the other hand, in [3] an iteration formula is obtained based on the standard Levenberg-Marquardt (LM) algorithm. By discarding a model term in the standard LM algorithm, an approximate iteration formula is derived, which is similar to that in [6]. For distinction, we call it approximate Levenberg-Marquardt ensemble randomized maximum likelihood (aLM-EnRML) algorithm. 1 These optimization algorithms are also applied to iterative EnKFs, e.g., in some of the aforementioned works. Title Suppressed Due to Excessive Length 3 In this work we show that an iteration formula similar to those used in the ES-MDA and RLM-MAC can be derived by applying the RLM to find an ensemble of solutions via solving a minimum-average-cost problem [21]. The gradient involved in the iteration is obtained in a way similar to the computation of the Kalman gain matrix in the EnKF. This derivation not only leads to an alternative theoretical tool in understanding and analyzing the behaviour of the aforementioned iES, but also provides insights and guidelines for further developments of the iES algorithm. As an example, we derive an alternative implementation of the iterative ES based on the RLM algorithm. Three numerical examples are then used to illustrate the performance of this new algorithm and compare it to the aLM-EnRML. Methodologies ES-MDA and aLM-EnRML Let m denote an m-dimensional reservoir model that contains the petrophysical properties to be estimated, g the reservoir simulator, and d a p-dimensional vector that contains all available observations in a certain time interval. Here d is assumed to contain certain measurement errors, with zero mean and covariance Cd. In the context of iterative ensemble smoothing, suppose that there is an ensemble M ≡ {mj} j=1 of Ne reservoir models available at the ith iteration step, then an iES updates M to its counterpart M ≡ {m j }e j=1 at the next iteration step via a certain iteration formula, which is the focus of our discussion below. For convenience of discussion later, let us define the following square root matrix Sm with respect to the model m (model square root for short): Sm = 1 √ Ne − 1 [ m1 − m, · · · ,miNe − m i ] , m = 1 Ne Ne ∑

Low-Rank Matrix Factorization under General Mixture Noise Distributions

Abstract: Many computer vision problems can be posed as learning a low-dimensional subspace from high dimensional data. The low rank matrix factorization (LRMF) represents a commonly utilized subspace learning strategy. Most of the current LRMF techniques are constructed on the optimization problem using L_1 norm and L_2 norm, which mainly deal with Laplacian and Gaussian noise, respectively. To make LRMF capable of adapting more complex noise, this paper proposes a new LRMF model by assuming noise as Mixture of Exponential Power (MoEP) distributions and proposes a penalized MoEP model by combining the penalized likelihood method with MoEP distributions. Such setting facilitates the learned LRMF model capable of automatically fitting the real noise through MoEP distributions. Each component in this mixture is adapted from a series of preliminary super-or sub-Gaussian candidates. An Expectation Maximization (EM) algorithm is also designed to infer the parameters involved in the proposed PMoEP model. The advantage of our method is demonstrated by extensive experiments on synthetic data, face modeling and hyperspectral image restoration.

Rayleigh-Rice Mixture Parameter Estimation via EM Algorithm for Change Detection in Multispectral Images

Abstract: The problem of estimating the parameters of a Rayleigh-Rice mixture density is often encountered in image analysis (e.g., remote sensing and medical image processing). In this paper, we address this general problem in the framework of change detection (CD) in multitemporal and multispectral images. One widely used approach to CD in multispectral images is based on the change vector analysis. Here, the distribution of the magnitude of the difference image can be theoretically modeled by a Rayleigh-Rice mixture density. However, given the complexity of this model, in applications, a Gaussian-mixture approximation is often considered, which may affect the CD results. In this paper, we present a novel technique for parameter estimation of the Rayleigh-Rice density that is based on a specific definition of the expectation-maximization algorithm. The proposed technique, which is characterized by good theoretical properties, iteratively updates the parameters and does not depend on specific optimization routines. Several numerical experiments on synthetic data demonstrate the effectiveness of the method, which is general and can be applied to any image processing problem involving the Rayleigh-Rice mixture density. In the CD context, the Rayleigh-Rice model (which is theoretically derived) outperforms other empirical models. Experiments on real multitemporal and multispectral remote sensing images confirm the validity of the model by returning significantly higher CD accuracies than those obtained by using the state-of-the-art approaches.

Structure learning of probabilistic logic programs by searching the clause space

Abstract: Learning probabilistic logic programming languages is receiving an increasing attention, and systems are available for learning the parameters (PRISM, LeProbLog, LFI-ProbLog and EMBLEM) or both structure and parameters (SEM-CP-logic and SLIPCASE) of these languages. In this paper we present the algorithm SLIPCOVER for “Structure LearnIng of Probabilistic logic programs by searChing OVER the clause space.” It performs a beam search in the space of probabilistic clauses and a greedy search in the space of theories using the log likelihood of the data as the guiding heuristics. To estimate the log likelihood, SLIPCOVER performs Expectation Maximization with EMBLEM. The algorithm has been tested on five real world datasets and compared with SLIPCASE, SEM-CP-logic, Aleph and two algorithms for learning Markov Logic Networks (Learning using Structural Motifs (LSM) and ALEPH++ExactL1). SLIPCOVER achieves higher areas under the precision-recall and receiver operating characteristic curves in most cases.

Online speech dereverberation using kalman filter and EM algorithm

Abstract: Speech signals recorded in a room are commonly degraded by reverberation. In most cases, both the speech signal and the acoustic system of the room are unknown and time-varying. In this paper, a scenario with a single desired sound source and slowly time-varying and spatially-white noise is considered, and a multi-microphone algorithm that simultaneously estimates the clean speech signal and the time-varying acoustic system is proposed. The recursive expectation-maximization scheme is employed to obtain both the clean speech signal and the acoustic system in an online manner. In the expectation step, the Kalman filter is applied to extract a new sample of the clean signal, and in the maximization step, the system estimate is updated according to the output of the Kalman filter. Experimental results show that the proposed method is able to significantly reduce reverberation and increase the speech quality. Moreover, the tracking ability of the algorithm was validated in practical scenarios using human speakers moving in a natural manner.

EM for mixtures

Abstract: Maximum likelihood through the EM algorithm is widely used to estimate the parameters in hidden structure models such as Gaussian mixture models. But the EM algorithm has well-documented drawbacks: its solution could be highly dependent from its initial position and it may fail as a result of degeneracies. We stress the practical dangers of theses limitations and how carefully they should be dealt with. Our main conclusion is that no method enables to address them satisfactory in all situations. But improvements are introduced, first, using a penalized log-likelihood of Gaussian mixture models in a Bayesian regularization perspective and, second, choosing the best among several relevant initialisation strategies. In this perspective, we also propose new recursive initialization strategies which prove helpful. They are compared with standard initialization procedures through numerical experiments and their effects on model selection criteria are analyzed.

Clustering algorithms for Stokes space modulation format recognition.

Abstract: Stokes space modulation format recognition (Stokes MFR) is a blind method enabling digital coherent receivers to infer modulation format information directly from a received polarization-division-multiplexed signal. A crucial part of the Stokes MFR is a clustering algorithm, which largely influences the performance of the detection process, particularly at low signal-to-noise ratios. This paper reports on an extensive study of six different clustering algorithms: k-means, expectation maximization, density-based DBSCAN and OPTICS, spectral clustering and maximum likelihood clustering, used for discriminating between dual polarization: BPSK, QPSK, 8-PSK, 8-QAM, and 16-QAM. We determine essential performance metrics for each clustering algorithm and modulation format under test: minimum required signal-to-noise ratio, detection accuracy and algorithm complexity.

The Generalized Linear Mixed Cluster-Weighted Model

Abstract: Cluster-weighted models (CWMs) are a flexible family of mixture models for fitting the joint distribution of a random vector composed of a response variable and a set of covariates. CWMs act as a convex combination of the products of the marginal distribution of the covariates and the conditional distribution of the response given the covariates. In this paper, we introduce a broad family of CWMs in which the component conditional distributions are assumed to belong to the exponential family and the covariates are allowed to be of mixed-type. Under the assumption of Gaussian covariates, sufficient conditions for model identifiability are provided. Moreover, maximum likelihood parameter estimates are derived using the EM algorithm. Parameter recovery, classification assessment, and performance of some information criteria are investigated through a broad simulation design. An application to real data is finally presented, with the proposed model outperforming other well-established mixture-based approaches.

Expectation–Maximization Approach to Fault Diagnosis With Missing Data

Abstract: This paper introduces a data-driven approach for fault diagnosis in the presence of incomplete monitor data. The expectation–maximization (EM) algorithm is applied to handle missing data in order to obtain a maximum-likelihood solution for the discrete (or categorical) distribution. Because of the nature of categorical distributions, the maximization step of the EM algorithm is shown in this paper to have an easily calculated analytical solution, making this method computationally simple. An experimental study on a ball-and-tube system is investigated to demonstrate advantages of the proposed approach.

Likelihood‐based dynamic factor analysis for measurement and forecasting

Abstract: Summary We present new results for the likelihood-based analysis of the dynamic factor model. The latent factors are modelled by linear dynamic stochastic processes. The idiosyncratic disturbance series are specified as autoregressive processes with mutually correlated innovations. The new results lead to computationally efficient procedures for the estimation of the factors and for the parameter estimation by maximum likelihood methods. We also present the implications of our results for models with regression effects, for Bayesian analysis, for signal extraction, and for forecasting. An empirical illustration is provided for the analysis of a large panel of macroeconomic time series.

High-dimensional regression with gaussian mixtures and partially-latent response variables

Abstract: The problem of approximating high-dimensional data with a low-dimensional representation is addressed. The article makes the following contributions. An inverse regression framework is proposed, which exchanges the roles of input and response, such that the low-dimensional variable becomes the regressor, and which is tractable. A mixture of locally-linear probabilistic mapping model is introduced, that starts with estimating the parameters of the inverse regression, and follows with inferring closed-form solutions for the forward parameters of the high-dimensional regression problem of interest. Moreover, a partially-latent paradigm is introduced, such that the vector-valued response variable is composed of both observed and latent entries, thus being able to deal with data contaminated by experimental artifacts that cannot be explained with noise models. The proposed probabilistic formulation could be viewed as a latent-variable augmentation of regression. Expectation-maximization (EM) procedures are introduced, based on a data augmentation strategy which facilitates the maximum-likelihood search over the model parameters. Two augmentation schemes are proposed and the associated EM inference procedures are described in detail; they may well be viewed as generalizations of a number of EM regression, dimension reduction, and factor analysis algorithms. The proposed framework is validated with both synthetic and real data. Experimental evidence is provided that the method outperforms several existing regression techniques.

Sparse estimation via nonconcave penalized likelihood in factor analysis model

Abstract: We consider the problem of sparse estimation in a factor analysis model. A traditional estimation procedure in use is the following two-step approach: the model is estimated by maximum likelihood method and then a rotation technique is utilized to find sparse factor loadings. However, the maximum likelihood estimates cannot be obtained when the number of variables is much larger than the number of observations. Furthermore, even if the maximum likelihood estimates are available, the rotation technique does not often produce a sufficiently sparse solution. In order to handle these problems, this paper introduces a penalized likelihood procedure that imposes a nonconvex penalty on the factor loadings. We show that the penalized likelihood procedure can be viewed as a generalization of the traditional two-step approach, and the proposed methodology can produce sparser solutions than the rotation technique. A new algorithm via the EM algorithm along with coordinate descent is introduced to compute the entire solution path, which permits the application to a wide variety of convex and nonconvex penalties. Monte Carlo simulations are conducted to investigate the performance of our modeling strategy. A real data example is also given to illustrate our procedure.

Image Outlier Detection and Feature Extraction via L1-Norm-Based 2D Probabilistic PCA

Abstract: This paper introduces an L1-norm-based probabilistic principal component analysis model on 2D data (L1-2DPPCA) based on the assumption of the Laplacian noise model. The Laplacian or L1 density function can be expressed as a superposition of an infinite number of Gaussian distributions. Under this expression, a Bayesian inference can be established based on the variational expectation maximization approach. All the key parameters in the probabilistic model can be learned by the proposed variational algorithm. It has experimentally been demonstrated that the newly introduced hidden variables in the superposition can serve as an effective indicator for data outliers. Experiments on some publicly available databases show that the performance of L1-2DPPCA has largely been improved after identifying and removing sample outliers, resulting in more accurate image reconstruction than the existing PCA-based methods. The performance of feature extraction of the proposed method generally outperforms other existing algorithms in terms of reconstruction errors and classification accuracy.

MLMD: Maximum Likelihood Mixture Decoupling for Fast and Accurate Point Cloud Registration

Abstract: Registration of Point Cloud Data (PCD) forms a core component of many 3D vision algorithms such as object matching and environment reconstruction. In this paper, we introduce a PCD registration algorithm that utilizes Gaussian Mixture Models (GMM) and a novel dual-mode parameter optimization technique which we call mixture decoupling. We show how this decoupling technique facilitates both faster and more robust registration by first optimizing over the mixture parameters (decoupling the mixture weights, means, and co variances from the points) before optimizing over the 6 DOF registration parameters. Furthermore, we frame both the decoupling and registration process inside a unified, dual-mode Expectation Maximization (EM) framework, for which we derive a Maximum Likelihood Estimation (MLE) solution along with a parallel implementation on the GPU. We evaluate our MLE-based mixture decoupling (MLMD) registration method over both synthetic and real data, showing better convergence for a wider range of initial conditions and higher speeds than previous state of the art methods.

Application of Machine Learning Techniques for Amplitude and Phase Noise Characterization

Abstract: In this paper, tools from machine learning community, such as Bayesian filtering and expectation maximization parameter estimation, are presented and employed for laser amplitude and phase noise characterization. We show that phase noise estimation based on Bayesian filtering outperforms conventional time-domain approach in the presence of moderate measurement noise. Additionally, carrier synchronization based on Bayesian filtering, in combination with expectation maximization, is demonstrated for the first time experimentally.

Missing in Asynchronicity: A Kalman-em Approach for Multivariate Realized Covariance Estimation

Abstract: Motivated by the need for a positive-semidefinite estimator of multivariate realized covariance matrices, we model noisy and asynchronous ultra-high-frequency asset prices in a state-space framework with missing data We then estimate the covariance matrix of the latent states through a Kalman smoother and expectation maximization (KEM) algorithm Iterating between the two EM steps, we obtain a covariance matrix estimate which is robust to both asynchronicity and microstructure noise, and positive-semidefinite by construction We show the performance of the KEM estimator using extensive Monte Carlo simulations that mimic the liquidity and market microstructure characteristics of the S&P 500 universe as well as in a high-dimensional application on US stocks KEM provides very accurate covariance matrix estimates and significantly outperforms alternative approaches recently introduced in the literature

An EM algorithm for the estimation of parameters of a flexible cure rate model with generalized gamma lifetime and model discrimination using likelihood- and information-based methods

Abstract: In this paper, we consider the Conway---Maxwell Poisson (COM-Poisson) cure rate model based on a competing risks scenario. This model includes, as special cases, some of the well-known cure rate models discussed in the literature. By assuming the time-to-event to follow the generalized gamma distribution, which contains some of the commonly used lifetime distributions as special cases, we develop exact likelihood inference based on the expectation maximization algorithm. The standard errors of the maximum likelihood estimates are obtained by inverting the observed information matrix. An extensive Monte Carlo simulation study is performed to examine the method of inference developed here. Model discrimination within the generalized gamma family is also carried out by means of likelihood- and information-based methods to select the particular lifetime distribution that provides an adequate fit to the data. Finally, a data on cancer recurrence is analyzed to illustrate the flexibility of the COM-Poisson family and the generalized gamma family so as to select a parsimonious competing cause distribution and a lifetime distribution that jointly provide an adequate fit to the data.

Reliability Modeling and Life Estimation Using an Expectation Maximization Based Wiener Degradation Model for Momentum Wheels

Abstract: The momentum wheel (MW) plays a significant role in ensuring the success of satellite missions, the reliability information of MW can be provided by collecting degradation data when there exists certain performance characteristics that degrade over time. In this paper, we develop a reliability modeling and life estimation approach for MW used in satellites based on the expectation maximization (EM) algorithm from a Wiener degradation model. The degradation model corresponding to a Wiener process with the random effect is first established using failure modes, mechanisms, and effects analysis. Afterwards, the first hitting time is employed to describe the failure time, and the explicit result of the reliability function is derived in terms of the Wiener degradation model. As the likelihood function for such a model contains unobserved latent variables, an EM algorithm is adopted to obtain the maximum likelihood estimators of model parameters efficiently. Finally, the effectiveness of the developed approach is validated using the degradation data from a specific type of MW.

Scalable estimation strategies based on stochastic approximations: classical results and new insights

Abstract: Estimation with large amounts of data can be facilitated by stochastic gradient methods, in which model parameters are updated sequentially using small batches of data at each step. Here, we review early work and modern results that illustrate the statistical properties of these methods, including convergence rates, stability, and asymptotic bias and variance. We then overview modern applications where these methods are useful, ranging from an online version of the EM algorithm to deep learning. In light of these results, we argue that stochastic gradient methods are poised to become benchmark principled estimation procedures for large datasets, especially those in the family of stable proximal methods, such as implicit stochastic gradient descent.