Statistics for Spatial Data.

We consider the problem of embedding entities and relationships of multi-relational data in low-dimensional vector spaces. Our objective is to propose a canonical model which is easy to train, contains a reduced number of parameters and can scale up to very large databases. Hence, we propose TransE, a method which models relationships by interpreting them as translations operating on the low-dimensional embeddings of the entities. Despite its simplicity, this assumption proves to be powerful since extensive experiments show that TransE significantly outperforms state-of-the-art methods in link prediction on two knowledge bases. Besides, it can be successfully trained on a large scale data set with 1M entities, 25k relationships and more than 17M training samples.

/pdf/translating-embeddings-for-modeling-multi-relational-data-k0qom0snwv.pdf

Translating Embeddings for Modeling Multi-relational Data

Preface to the revised edition Remarks on notation 1. Basic examples 2. Characterization and existence 3. Stable processes and their extensions 4. The Levy-Ito decomposition of sample functions 5. Distributional properties of Levy processes 6. Subordination and density transformation 7. Recurrence and transience 8. Potential theory for Levy processes 9. Wiener-Hopf factorizations 10. More distributional properties Supplement Solutions to exercises References and author index Subject index.

Lévy processes and infinitely divisible distributions

How can a machine learn from experience? Probabilistic modelling provides a framework for understanding what learning is, and has therefore emerged as one of the principal theoretical and practical approaches for designing machines that learn from data acquired through experience. The probabilistic framework, which describes how to represent and manipulate uncertainty about models and predictions, has a central role in scientific data analysis, machine learning, robotics, cognitive science and artificial intelligence. This Review provides an introduction to this framework, and discusses some of the state-of-the-art advances in the field, namely, probabilistic programming, Bayesian optimization, data compression and automatic model discovery.

/pdf/probabilistic-machine-learning-and-artificial-intelligence-4gu82j2v4w.pdf

Probabilistic machine learning and artificial intelligence

Graph embedding algorithms embed a graph into a vector space where the structure and the inherent properties of the graph are preserved The existing graph embedding methods cannot preserve the asymmetric transitivity well, which is a critical property of directed graphs Asymmetric transitivity depicts the correlation among directed edges, that is, if there is a directed path from u to v, then there is likely a directed edge from u to v Asymmetric transitivity can help in capturing structures of graphs and recovering from partially observed graphs To tackle this challenge, we propose the idea of preserving asymmetric transitivity by approximating high-order proximity which are based on asymmetric transitivity In particular, we develop a novel graph embedding algorithm, High-Order Proximity preserved Embedding (HOPE for short), which is scalable to preserve high-order proximities of large scale graphs and capable of capturing the asymmetric transitivity More specifically, we first derive a general formulation that cover multiple popular high-order proximity measurements, then propose a scalable embedding algorithm to approximate the high-order proximity measurements based on their general formulation Moreover, we provide a theoretical upper bound on the RMSE (Root Mean Squared Error) of the approximation Our empirical experiments on a synthetic dataset and three real-world datasets demonstrate that HOPE can approximate the high-order proximities significantly better than the state-of-art algorithms and outperform the state-of-art algorithms in tasks of reconstruction, link prediction and vertex recommendation

Asymmetric Transitivity Preserving Graph Embedding

As the availability and importance of relational data—such as the friendships summarized on a social networking website—increases, it becomes increasingly important to have good models for such data. The kinds of latent structure that have been considered for use in predicting links in such networks have been relatively limited. In particular, the machine learning community has focused on latent class models, adapting Bayesian nonparametric methods to jointly infer how many latent classes there are while learning which entities belong to each class. We pursue a similar approach with a richer kind of latent variable—latent features—using a Bayesian nonparametric approach to simultaneously infer the number of features at the same time we learn which entities have each feature. Our model combines these inferred features with known covariates in order to perform link prediction. We demonstrate that the greater expressiveness of this approach allows us to improve performance on three datasets.

/pdf/nonparametric-latent-feature-models-for-link-prediction-qpfy14dv10.pdf

Nonparametric Latent Feature Models for Link Prediction

The Indian Buet Process (IBP) is a nonparametric prior for latent feature models in which observations are influenced by a combination of hidden features. For example, images may be composed of several objects and sounds may consist of several notes. Latent feature models seek to infer these unobserved features from a set of observations; the IBP provides a principled prior in situations where the number of hidden features is unknown. Current inference methods for the IBP have all relied on sampling. While these methods are guaranteed to be accurate in the limit, samplers for the IBP tend to mix slowly in practice. We develop a deterministic variational method for inference in the IBP based on a truncated stick-breaking approximation, provide theoretical bounds on the truncation error, and evaluate our method in several data regimes.

/pdf/variational-inference-for-the-indian-buffet-process-a6ktdl4fd9.pdf

Variational Inference for the Indian Buffet Process

Priors for Bayesian nonparametric latent feature models were originally developed a little over five years ago, sparking interest in a new type of Bayesian nonparametric model. Since then, there have been three main areas of research for people interested in these priors: extensions/generalizations of the priors, inference algorithms, and applications. This dissertation summarizes our work advancing the state of the art in all three of these areas. In the first area, we present a non-exchangeable framework for generalizing and extending the original priors, allowing more prior knowledge to be used in nonparametric priors. Within this framework, we introduce four concrete generalizations that are applicable when we have prior knowledge about object relationships that can be captured either via a tree or chain. We discuss how to develop and derive these priors as well as how to perform posterior inference in models using them. In the area of inference algorithms, we present the first variational approximation for one class of these priors, demonstrating in what regimes they might be preferred over more traditional MCMC approaches. Finally, we present an application of basic nonparametric latent features models to link prediction as well as applications of our non-exchangeable priors to tree-structured choice models and human genomic data.

/pdf/bayesian-nonparametric-latent-feature-models-27y0rm65an.pdf

Bayesian nonparametric latent feature models

Nonparametric Bayesian models are often based on the assumption that the objects being modeled are exchangeable. While appropriate in some applications (e.g., bag-of-words models for documents), exchangeability is sometimes assumed simply for computational reasons; non-exchangeable models might be a better choice for applications based on subject matter. Drawing on ideas from graphical models and phylogenetics, we describe a non-exchangeable prior for a class of nonparametric latent feature models that is nearly as efficient computationally as its exchangeable counterpart. Our model is applicable to the general setting in which the dependencies between objects can be expressed using a tree, where edge lengths indicate the strength of relationships. We demonstrate an application to modeling probabilistic choice.

/pdf/the-phylogenetic-indian-buffet-process-a-non-exchangeable-s5roagr9na.pdf

The phylogenetic Indian Buffet process: a non-exchangeable nonparametric prior for latent features

This paper presents a VLSI-friendly linear optic flow sensor capable of being implemented on a single chip. This sensor belongs to the "token method" class of optic flow sensors: motion is measured by tracing the movement of a "token" or a feature across the focal plane. Specially shaped photoreceptor circuits and simple analog circuitry implement an array of feature detectors on the focal plane chip. The determination of a feature's location in the focal plane is performed using a winner-take-all (WTA) circuit. Motion is detected by using compact logic circuitry to detect appropriate movements of the high WTA output. Speed measurement is performed by timing how long an edge stays at one location before moving to the next.

Kurt T. Miller

Papers

Nonparametric Latent Feature Models for Link Prediction

Variational Inference for the Indian Buffet Process

Bayesian nonparametric latent feature models

The phylogenetic Indian Buffet process: a non-exchangeable nonparametric prior for latent features

Feature tracking linear optic flow sensor chip