Probabilistic Latent Semantic Indexing is a novel approach to automated document indexing which is based on a statistical latent class model for factor analysis of count data. Fitted from a training corpus of text documents by a generalization of the Expectation Maximization algorithm, the utilized model is able to deal with domain{specific synonymy as well as with polysemous words. In contrast to standard Latent Semantic Indexing (LSI) by Singular Value Decomposition, the probabilistic variant has a solid statistical foundation and defines a proper generative data model. Retrieval experiments on a number of test collections indicate substantial performance gains over direct term matching methods as well as over LSI. In particular, the combination of models with different dimensionalities has proven to be advantageous.

https://www.researchgate.net/profile/Thomas_Hofmann10/publication/2941307_Probabilistic_Latent_Semantic_Indexing/links/548428f50cf25dbd59eb12ed.pdf

Probabilistic latent semantic indexing

When a parameter space has a certain underlying structure, the ordinary gradient of a function does not represent its steepest direction, but the natural gradient does. Information geometry is used for calculating the natural gradients in the parameter space of perceptrons, the space of matrices (for blind source separation), and the space of linear dynamical systems (for blind source deconvolution). The dynamical behavior of natural gradient online learning is analyzed and is proved to be Fisher efficient, implying that it has asymptotically the same performance as the optimal batch estimation of parameters. This suggests that the plateau phenomenon, which appears in the backpropagation learning algorithm of multilayer perceptrons, might disappear or might not be so serious when the natural gradient is used. An adaptive method of updating the learning rate is proposed and analyzed.

Natural gradient works efficiently in learning

We review machine learning methods employing positive definite kernels. These methods formulate learning and estimation problems in a reproducing kernel Hilbert space (RKHS) of functions defined on the data domain, expanded in terms of a kernel. Working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions. The latter include nonlinear functions as well as functions defined on nonvectorial data. We cover a wide range of methods, ranging from binary classifiers to sophisticated methods for estimation with structured data.

/pdf/kernel-methods-in-machine-learning-2d4c18y756.pdf

Kernel methods in machine learning

Type iib superstrings, bps monopoles, and three-dimensional gauge dynamics

The paper proposes Metropolis adjusted Langevin and Hamiltonian Monte Carlo sampling methods defined on the Riemann manifold to resolve the shortcomings of existing Monte Carlo algorithms when sampling from target densities that may be high dimensional and exhibit strong correlations. The methods provide fully automated adaptation mechanisms that circumvent the costly pilot runs that are required to tune proposal densities for Metropolis–Hastings or indeed Hamiltonian Monte Carlo and Metropolis adjusted Langevin algorithms. This allows for highly efficient sampling even in very high dimensions where different scalings may be required for the transient and stationary phases of the Markov chain. The methodology proposed exploits the Riemann geometry of the parameter space of statistical models and thus automatically adapts to the local structure when simulating paths across this manifold, providing highly efficient convergence and exploration of the target density. The performance of these Riemann manifold Monte Carlo methods is rigorously assessed by performing inference on logistic regression models, log-Gaussian Cox point processes, stochastic volatility models and Bayesian estimation of dynamic systems described by non-linear differential equations. Substantial improvements in the time-normalized effective sample size are reported when compared with alternative sampling approaches. MATLAB code that is available from http://www.ucl.ac.uk/statistics/research/rmhmc allows replication of all the results reported.

/pdf/riemann-manifold-langevin-and-hamiltonian-monte-carlo-254effw0xm.pdf

Riemann manifold Langevin and Hamiltonian Monte Carlo methods

In this note we introduce the notion of bundle gerbe K-theory and investigate the relation to twisted K-theory. We provide some examples. Possible applications of bundle gerbe K-theory to the classification of K-brane charges in nontrivial backgrounds are briefly discussed.

/pdf/twisted-k-theory-and-k-theory-of-bundle-gerbes-3l72pcmscl.pdf

Twisted K-theory and K-Theory of Bundle Gerbes

Just as $\Cstar$ principal bundles provide a geometric realisation of two-dimensional integral cohomology; gerbes or sheaves of groupoids, provide a geometric realisation of three dimensional integral cohomology through their Dixmier-Douady class. I consider an alternative, related, geometric realisation of three dimensional cohomology called a bundle gerbe. Every bundle gerbe gives rise to a gerbe and most of the well-known examples examples of gerbes are bundle gerbes. I discuss the properties of bundle gerbes, in particular bundle gerbe connections and curvature and their associated Dixmier-Douady class.

Bundle gerbes

Several years ago our statistical friends and relations introduced us to the work of Amari and Barndorff-Nielsen on applications of differential geometry to statistics. This book has arisen because we believe that there is a deep relationship between statistics and differential geometry and moreoever that this relationship uses parts of differential geometry, particularly its 'higher-order' aspects not readily accessible to a statistical audience from the existing literature. It is, in part, a long reply to the frequent requests we have had for references on differential geometry! While we have not gone beyond the path-breaking work of Amari and Barndorff- Nielsen in the realm of applications, our book gives some new explanations of their ideas from a first principles point of view as far as geometry is concerned. In particular it seeks to explain why geometry should enter into parametric statistics, and how the theory of asymptotic expansions involves a form of higher-order differential geometry. The first chapter of the book explores exponential families as flat geometries. Indeed the whole notion of using log-likelihoods amounts to exploiting a particular form of flat space known as an affine geometry, in which straight lines and planes make sense, but lengths and angles are absent. We use these geometric ideas to introduce the notion of the second fundamental form of a family whose vanishing characterises precisely the exponential families.

Differential Geometry and Statistics

The notion of stable isomorphism of bundle gerbes is considered. It has the consequence that the stable isomorphism classes of bundle gerbes over a manifold M are in bijective correspondence with H3(M, ℤ). Stable isomorphism sheds light on the local theory of bundle gerbes and enables a classifying theory for bundle gerbes to be developed using results of Gajer on B[Copf ]× bundles.

Bundle Gerbes: Stable Isomorphism and Local Theory

ForG a classical group, an equivalence is exhibited between:

A)

G monopoles over ℝ3, with maximal symmetry breaking at infinity,




B)

families of (rank (G)) algebraic curves inTℙ1, along with divisors on those curves, satisfying certain constraints,




C)

solutions of Nahm's equations over (rank(G)) intervals, satisfying the appropriate boundary conditions.






A) and B) are linked by twistor techniques, B) and C) via the Krichever method for solving non-linear differential equations, and A) and C) via the ADHMN construction, providing a unified picture of techniques for solution. Amongst other things, an asymptotic formula for the Higgs field of the monopole is computed.

/pdf/on-the-construction-of-monopoles-for-the-classical-groups-1fffwyhulo.pdf

Michael K. Murray

Papers

Twisted K-theory and K-Theory of Bundle Gerbes

Bundle gerbes

Differential Geometry and Statistics

Bundle Gerbes: Stable Isomorphism and Local Theory

On the construction of monopoles for the classical groups