Journal ArticleDOI
Information geometry of the EM and em algorithms for neural networks
Reads0
Chats0
TLDR
A unified information geometrical framework for studying stochastic models of neural networks, by focusing on the EM and em algorithms, and proves a condition that guarantees their equivalence.About:
This article is published in Neural Networks.The article was published on 1995-12-16. It has received 339 citations till now. The article focuses on the topics: Stochastic neural network & Mixture model.read more
Citations
More filters
Journal ArticleDOI
Hierarchical mixtures of experts and the EM algorithm
TL;DR: An Expectation-Maximization (EM) algorithm for adjusting the parameters of the tree-structured architecture for supervised learning and an on-line learning algorithm in which the parameters are updated incrementally.
Proceedings ArticleDOI
Clustering with Bregman Divergences
TL;DR: This paper proposes and analyzes parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergences, and shows that there is a bijection between regular exponential families and a largeclass of BRegman diverGences, that is called regular Breg man divergence.
Journal ArticleDOI
Exponentiated gradient versus gradient descent for linear predictors
Jyrki Kivinen,Manfred K. Warmuth +1 more
TL;DR: The bounds suggest that the losses of the algorithms are in general incomparable, but EG(+/-) has a much smaller loss if only a few components of the input are relevant for the predictions, which is quite tight already on simple artificial data.
Journal ArticleDOI
Clustering on the Unit Hypersphere using von Mises-Fisher Distributions
TL;DR: A generative mixture-model approach to clustering directional data based on the von Mises-Fisher distribution, which arises naturally for data distributed on the unit hypersphere, and derives and analyzes two variants of the Expectation Maximization framework for estimating the mean and concentration parameters of this mixture.
References
More filters
Journal ArticleDOI
Maximum likelihood from incomplete data via the EM algorithm
Journal ArticleDOI
Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
Stuart Geman,Donald Geman +1 more
TL;DR: The analogy between images and statistical mechanics systems is made and the analogous operation under the posterior distribution yields the maximum a posteriori (MAP) estimate of the image given the degraded observations, creating a highly parallel ``relaxation'' algorithm for MAP estimation.
Journal ArticleDOI
Linear Statistical Inference and its Applications
P. G. Moore,C. Radhakrishna Rao +1 more
TL;DR: The theory of least squares and analysis of variance has been studied in the literature for a long time, see as mentioned in this paper for a review of some of the most relevant works. But the main focus of this paper is on the analysis of variance.
Journal ArticleDOI
A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains
Journal ArticleDOI
Adaptive mixtures of local experts
TL;DR: A new supervised learning procedure for systems composed of many separate networks, each of which learns to handle a subset of the complete set of training cases, which is demonstrated to be able to be solved by a very simple expert network.