Open AccessProceedings Article
Poincaré Embeddings for Learning Hierarchical Representations
Maximillian Nickel,Douwe Kiela +1 more
- Vol. 30, pp 6338-6347
Reads0
Chats0
TLDR
This work introduces a new approach for learning hierarchical representations of symbolic data by embedding them into hyperbolic space -- or more precisely into an n-dimensional Poincare ball -- and introduces an efficient algorithm to learn the embeddings based on Riemannian optimization.Abstract:
Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, state-of-the-art embedding methods typically do not account for latent hierarchical structures which are characteristic for many complex symbolic datasets. In this work, we introduce a new approach for learning hierarchical representations of symbolic data by embedding them into hyperbolic space -- or more precisely into an n-dimensional Poincare ball. Due to the underlying hyperbolic geometry, this allows us to learn parsimonious representations of symbolic data by simultaneously capturing hierarchy and similarity. We present an efficient algorithm to learn the embeddings based on Riemannian optimization and show experimentally that Poincare embeddings can outperform Euclidean embeddings significantly on data with latent hierarchies, both in terms of representation capacity and in terms of generalization ability.read more
Citations
More filters
Posted Content
Geom-GCN: Geometric Graph Convolutional Networks
TL;DR: The proposed aggregation scheme is permutation-invariant and consists of three modules, node embedding, structural neighborhood, and bi-level aggregation, and an implementation of the scheme in graph convolutional networks, termed Geom-GCN, to perform transductive learning on graphs.
Proceedings ArticleDOI
Mitigating Gender Bias in Natural Language Processing: Literature Review
Tony Sun,Andrew Gaut,Shirlyn Tang,Yuxin Huang,Mai ElSherief,Jieyu Zhao,Diba Mirza,Elizabeth Belding,Kai-Wei Chang,William Yang Wang +9 more
TL;DR: This paper discusses gender bias based on four forms of representation bias and analyzes methods recognizing gender bias in NLP, and discusses the advantages and drawbacks of existing gender debiasing methods.
Posted Content
Hyperbolic Graph Convolutional Neural Networks
TL;DR: Hyperbolic Graph Convolutional Neural Network (HGCN) as discussed by the authors is the first inductive hyperbolic GCN that leverages both the expressiveness of GCNs and hyperboloid geometry to learn inductive node representations for hierarchical and scale-free graphs.
Posted Content
Multi-Object Representation Learning with Iterative Variational Inference
Klaus Greff,Raphael Lopez Kaufman,Rishabh Kabra,Nicholas Watters,Christopher P. Burgess,Daniel Zoran,Loic Matthey,Matthew Botvinick,Alexander Lerchner +8 more
TL;DR: In this paper, the authors argue for the importance of learning to segment and represent objects jointly, and demonstrate that, starting from the simple assumption that a scene is composed of multiple entities, it is possible to learn to segment images into interpretable objects with disentangled representations.
Posted Content
Visualizing and Measuring the Geometry of BERT
TL;DR: This paper describes qualitative and quantitative investigations of one particularly effective model, BERT, and finds evidence of a fine-grained geometric representation of word senses in both attention matrices and individual word embeddings.