Open AccessProceedings Article
What graph neural networks cannot learn: depth vs width
Reads0
Chats0
TLDR
GNNmp are shown to be Turing universal under sufficient conditions on their depth, width, node attributes, and layer expressiveness, and it is discovered that GNNmp can lose a significant portion of their power when their depth and width is restricted.Abstract:
This paper studies theoretically the capacity limits of graph neural networks (GNN) falling within the message-passing framework. Two main results are presented. First, GNN are shown to be Turing universal under sufficient conditions on their depth, width, node identification, and layer expressiveness. Second, it is discovered that GNN can lose a significant portion of their power when their depth and width is restricted. The proposed impossibility statements stem from a new technique that enables the repurposing of seminal results from theoretical computer science and leads to lower bounds for an array of decision, optimization, and estimation problems involving graphs. Strikingly, several of these problems are deemed impossible unless the product of a GNN's depth and width exceeds (a function of) the graph size; this dependence remains significant even for tasks that appear simple or when considering approximation.read more
Citations
More filters
Posted Content
Graph Neural Networks: A Review of Methods and Applications
Jie Zhou,Ganqu Cui,Shengding Hu,Zhengyan Zhang,Cheng Yang,Zhiyuan Liu,Lifeng Wang,Changcheng Li,Maosong Sun +8 more
TL;DR: A detailed review over existing graph neural network models is provided, systematically categorize the applications, and four open problems for future research are proposed.
Journal ArticleDOI
Graph Neural Networks: A Review of Methods and Applications
Jie Zhou,Ganqu Cui,Shengding Hu,Zhengyan Zhang,Cheng Yang,Zhiyuan Liu,Lifeng Wang,Changcheng Li,Maosong Sun +8 more
TL;DR: In this paper, the authors propose a general design pipeline for GNN models and discuss the variants of each component, systematically categorize the applications, and propose four open problems for future research.
Journal Article
Benchmarking Graph Neural Networks
TL;DR: A reproducible GNN benchmarking framework is introduced, with the facility for researchers to add new models conveniently for arbitrary datasets, and a principled investigation into the recent Weisfeiler-Lehman GNNs (WL-GNNs) compared to message passing-based graph convolutional networks (GCNs).
Journal Article
Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
TL;DR: Graph Substructure Networks (GSN) is proposed, a topologically-aware message passing scheme based on substructure encoding that allows for multiple attractive properties of standard GNNs such as locality and linear network complexity, while being able to disambiguate even hard instances of graph isomorphism.
Posted Content
Machine Learning on Graphs: A Model and Comprehensive Taxonomy
TL;DR: A comprehensive taxonomy of representation learning methods for graph-structured data is proposed, aiming to unify several disparate bodies of work and provide a solid foundation for understanding the intuition behind these methods, and enables future research in the area.
References
More filters
Journal ArticleDOI
Emergence of Scaling in Random Networks
TL;DR: A model based on these two ingredients reproduces the observed stationary scale-free distributions, which indicates that the development of large networks is governed by robust self-organizing phenomena that go beyond the particulars of the individual systems.
Journal ArticleDOI
Multilayer feedforward networks are universal approximators
TL;DR: It is rigorously established that standard multilayer feedforward networks with as few as one hidden layer using arbitrary squashing functions are capable of approximating any Borel measurable function from one finite dimensional space to another to any desired degree of accuracy, provided sufficiently many hidden units are available.
Posted Content
Semi-Supervised Classification with Graph Convolutional Networks
Thomas Kipf,Max Welling +1 more
TL;DR: A scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs which outperforms related methods by a significant margin.
Journal ArticleDOI
Approximation by superpositions of a sigmoidal function
TL;DR: It is demonstrated that finite linear combinations of compositions of a fixed, univariate function and a set of affine functionals can uniformly approximate any continuous function ofn real variables with support in the unit hypercube.
Journal ArticleDOI
Multilayer feedforward networks are universal approximators
HornikK.,StinchcombeM.,WhiteH. +2 more