Open AccessPosted Content
Graph Attention Networks
TLDR
Graph Attention Networks (GATs) as discussed by the authors leverage masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations.Abstract:
We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of costly matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset (wherein test graphs remain unseen during training).read more
Citations
More filters
Journal ArticleDOI
Dynamic Graph CNN for Learning on Point Clouds
TL;DR: This work proposes a new neural network module suitable for CNN-based high-level tasks on point clouds, including classification and segmentation called EdgeConv, which acts on graphs dynamically computed in each layer of the network.
Posted Content
Attention U-Net: Learning Where to Look for the Pancreas
Ozan Oktay,Jo Schlemper,Loic Le Folgoc,Matthew C. H. Lee,Mattias P. Heinrich,Kazunari Misawa,Kensaku Mori,Steven McDonagh,Nils Y. Hammerla,Bernhard Kainz,Ben Glocker,Daniel Rueckert +11 more
TL;DR: A novel attention gate (AG) model for medical imaging that automatically learns to focus on target structures of varying shapes and sizes is proposed to eliminate the necessity of using explicit external tissue/organ localisation modules of cascaded convolutional neural networks (CNNs).
Posted Content
Dynamic Graph CNN for Learning on Point Clouds
TL;DR: In this paper, a new neural network module called EdgeConv is proposed for CNN-based high-level tasks on point clouds including classification and segmentation, which is differentiable and can be plugged into existing architectures.
Journal ArticleDOI
Self-supervised Learning: Generative or Contrastive.
TL;DR: This survey takes a look into new self-supervised learning methods for representation in computer vision, natural language processing, and graph learning, and comprehensively review the existing empirical methods into three main categories according to their objectives.
Journal ArticleDOI
Graph convolutional networks: a comprehensive review
TL;DR: A comprehensive review specifically on the emerging field of graph convolutional networks, which is one of the most prominent graph deep learning models, is conducted and several open challenges are presented and potential directions for future research are discussed.
References
More filters
Proceedings ArticleDOI
Deep Residual Learning for Image Recognition
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
Journal ArticleDOI
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Journal ArticleDOI
Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles
Aravind Subramanian,Pablo Tamayo,Vamsi K. Mootha,Sayan Mukherjee,Benjamin L. Ebert,Michael A. Gillette,Amanda G. Paulovich,Scott L. Pomeroy,Todd R. Golub,Eric S. Lander,Jill P. Mesirov +10 more
TL;DR: The Gene Set Enrichment Analysis (GSEA) method as discussed by the authors focuses on gene sets, that is, groups of genes that share common biological function, chromosomal location, or regulation.
Journal Article
Dropout: a simple way to prevent neural networks from overfitting
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Journal Article
Visualizing Data using t-SNE
TL;DR: A new technique called t-SNE that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map, a variation of Stochastic Neighbor Embedding that is much easier to optimize, and produces significantly better visualizations by reducing the tendency to crowd points together in the center of the map.