scispace - formally typeset
Search or ask a question
Author

Yi Wu

Bio: Yi Wu is an academic researcher from Peking University. The author has contributed to research in topics: Computer science & Stability (learning theory). The author has an hindex of 1, co-authored 4 publications receiving 9 citations.

Papers
More filters
Proceedings ArticleDOI
TL;DR: Wang et al. as mentioned in this paper proposed a streaming GNN model based on continual learning so that the model is trained incrementally and up-to-date node representations can be obtained at each time step.
Abstract: Graph neural networks (GNNs) have achieved strong performance in various applications. In the real world, network data is usually formed in a streaming fashion. The distributions of patterns that refer to neighborhood information of nodes may shift over time. The GNN model needs to learn the new patterns that cannot yet be captured. But learning incrementally leads to the catastrophic forgetting problem that historical knowledge is overwritten by newly learned knowledge. Therefore, it is important to train GNN model to learn new patterns and maintain existing patterns simultaneously, which few works focus on. In this paper, we propose a streaming GNN model based on continual learning so that the model is trained incrementally and up-to-date node representations can be obtained at each time step. Firstly, we design an approximation algorithm to detect new coming patterns efficiently based on information propagation. Secondly, we combine two perspectives of data replaying and model regularization for existing pattern consolidation. Specially, a hierarchy-importance sampling strategy for nodes is designed and a weighted regularization term for GNN parameters is derived, achieving greater stability and generalization of knowledge consolidation. Our model is evaluated on real and synthetic data sets and compared with multiple baselines. The results of node classification prove that our model can efficiently update model parameters and achieve comparable performance to model retraining. In addition, we also conduct a case study on the synthetic data, and carry out some specific analysis for each part of our model, illustrating its ability to learn new knowledge and maintain existing knowledge from different perspectives.

40 citations

Proceedings ArticleDOI
19 Oct 2020
TL;DR: This paper proposes a streaming GNN model based on continual learning so that the model is trained incrementally and up-to-date node representations can be obtained at each time step, and designs an approximation algorithm to detect new coming patterns efficiently based on information propagation.
Abstract: Graph neural networks (GNNs) have achieved strong performance in various applications. In the real world, network data is usually formed in a streaming fashion. The distributions of patterns that refer to neighborhood information of nodes may shift over time. The GNN model needs to learn the new patterns that cannot yet be captured. But learning incrementally leads to the catastrophic forgetting problem that historical knowledge is overwritten by newly learned knowledge. Therefore, it is important to train GNN model to learn new patterns and maintain existing patterns simultaneously, which few works focus on. In this paper, we propose a streaming GNN model based on continual learning so that the model is trained incrementally and up-to-date node representations can be obtained at each time step. Firstly, we design an approximation algorithm to detect new coming patterns efficiently based on information propagation. Secondly, we combine two perspectives of data replaying and model regularization for existing pattern consolidation. Specially, a hierarchy-importance sampling strategy for nodes is designed and a weighted regularization term for GNN parameters is derived, achieving greater stability and generalization of knowledge consolidation. Our model is evaluated on real and synthetic data sets and compared with multiple baselines. The results of node classification prove that our model can efficiently update model parameters and achieve comparable performance to model retraining. In addition, we also conduct a case study on the synthetic data, and carry out some specific analysis for each part of our model, illustrating its ability to learn new knowledge and maintain existing knowledge from different perspectives.

30 citations

Journal ArticleDOI
TL;DR: The backpropagation neural network (BPNN) algorithm of artificial intelligence (AI) is utilized to predict A+H shares price for helping investors reduce the risk of stock investment, and a model that can predict multi-day stock prices is established.
Abstract: The backpropagation neural network (BPNN) algorithm of artificial intelligence (AI) is utilized to predict A+H shares price for helping investors reduce the risk of stock investment. First, the genetic algorithm (GA) is used to optimize BPNN, and a model that can predict multi-day stock prices is established. Then, the Principal Component Analysis (PCA) algorithm is introduced to improve the GA-BP model, aiming to provide a practical approach for analyzing the market risks of the A+H shares. The experimental results show that for A shares, the model has the best prediction effect on the price of Bank of China (BC), and the average prediction errors of opening price, maximum price, minimum price, as well as closing price are 0.0236, 0.0262, 0.0294 and 0.0339, respectively. For H shares, the model constructed has the best effect on the price prediction of China Merchants Bank (CMB). The average prediction errors of opening price, maximum price, minimum price and closing price are 0.0276, 0.0422, 0.0194 and 0.0619, respectively.

9 citations

Proceedings ArticleDOI
19 Apr 2021
TL;DR: GSKN1 as mentioned in this paper is a GNN model with a theoretically stronger ability to distinguish graph structures, based on anonymous walks (AWs), flexible substructure units, and derive it upon feature mappings of graph kernels (GKs).
Abstract: Graph neural networks (GNNs) have achieved tremendous success in graph mining. However, the inability of GNNs to model substructures in graphs remains a significant drawback. Specifically, message-passing GNNs (MPGNNs), as the prevailing type of GNNs, have been theoretically shown unable to distinguish, detect or count many graph substructures. While efforts have been paid to complement the inability, existing works either rely on pre-defined substructure sets, thus being less flexible, or are lacking in theoretical insights. In this paper, we propose GSKN1, a GNN model with a theoretically stronger ability to distinguish graph structures. Specifically, we design GSKN based on anonymous walks (AWs), flexible substructure units, and derive it upon feature mappings of graph kernels (GKs). We theoretically show that GSKN provably extends the 1-WL test, and hence the maximally powerful MPGNNs from both graph-level and node-level viewpoints. Correspondingly, various experiments are leveraged to evaluate GSKN, where GSKN outperforms a wide range of baselines, endorsing the analysis.

9 citations

Posted Content
TL;DR: GSKN as discussed by the authors is a GNN model with a theoretically stronger ability to distinguish graph structures, based on anonymous walks (AWs), flexible substructure units, and derive it upon feature mappings of graph kernels (GKs).
Abstract: Graph neural networks (GNNs) have achieved tremendous success in graph mining. However, the inability of GNNs to model substructures in graphs remains a significant drawback. Specifically, message-passing GNNs (MPGNNs), as the prevailing type of GNNs, have been theoretically shown unable to distinguish, detect or count many graph substructures. While efforts have been paid to complement the inability, existing works either rely on pre-defined substructure sets, thus being less flexible, or are lacking in theoretical insights. In this paper, we propose GSKN, a GNN model with a theoretically stronger ability to distinguish graph structures. Specifically, we design GSKN based on anonymous walks (AWs), flexible substructure units, and derive it upon feature mappings of graph kernels (GKs). We theoretically show that GSKN provably extends the 1-WL test, and hence the maximally powerful MPGNNs from both graph-level and node-level viewpoints. Correspondingly, various experiments are leveraged to evaluate GSKN, where GSKN outperforms a wide range of baselines, endorsing the analysis.

1 citations


Cited by
More filters
Proceedings ArticleDOI
22 Jan 2006
TL;DR: Some of the major results in random graphs and some of the more challenging open problems are reviewed, including those related to the WWW.
Abstract: We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

7,116 citations

Proceedings ArticleDOI
TL;DR: Wang et al. as mentioned in this paper proposed Spatial-Temporal Graph Ordinary Differential Equation Networks (STGODE) to capture spatial-temporal dynamics through a tensor-based ordinary differential equation (ODE), as well as a well-design temporal dialated convolution structure is used to capture long term temporal dependencies.
Abstract: Spatial-temporal forecasting has attracted tremendous attention in a wide range of applications, and traffic flow prediction is a canonical and typical example. The complex and long-range spatial-temporal correlations of traffic flow bring it to a most intractable challenge. Existing works typically utilize shallow graph convolution networks (GNNs) and temporal extracting modules to model spatial and temporal dependencies respectively. However, the representation ability of such models is limited due to: (1) shallow GNNs are incapable to capture long-range spatial correlations, (2) only spatial connections are considered and a mass of semantic connections are ignored, which are of great importance for a comprehensive understanding of traffic networks. To this end, we propose Spatial-Temporal Graph Ordinary Differential Equation Networks (STGODE). Specifically, we capture spatial-temporal dynamics through a tensor-based ordinary differential equation (ODE), as a result, deeper networks can be constructed and spatial-temporal features are utilized synchronously. To understand the network more comprehensively, semantical adjacency matrix is considered in our model, and a well-design temporal dialated convolution structure is used to capture long term temporal dependencies. We evaluate our model on multiple real-world traffic datasets and superior performance is achieved over state-of-the-art baselines.

85 citations

Posted Content
TL;DR: This paper constructs a new graph topology, called the feature graph, which takes features as new nodes and turns nodes into independent graphs and achieves much higher accuracy than the state-of-the-art methods in both data-incremental and class-increasing tasks.
Abstract: Graph neural networks are powerful models for many graph-structured tasks. In this paper, we aim to solve the problem of lifelong learning for graph neural networks. One of the main challenges is the effect of "catastrophic forgetting" for continuously learning a sequence of tasks, as the nodes can only be present to the model once. Moreover, the number of nodes changes dynamically in lifelong learning and this makes many graph models and sampling strategies inapplicable. To solve these problems, we construct a new graph topology, called the feature graph. It takes features as new nodes and turns nodes into independent graphs. This successfully converts the original problem of node classification to graph classification. In this way, the increasing nodes in lifelong learning can be regarded as increasing training samples, which makes lifelong learning easier. We demonstrate that the feature graph achieves much higher accuracy than the state-of-the-art methods in both data-incremental and class-incremental tasks. We expect that the feature graph will have broad potential applications for graph-structured tasks in lifelong learning.

26 citations

Proceedings ArticleDOI
14 Aug 2022
TL;DR: This work first adopts the masked edge prediction, the most simplest and popular pretext task, to pre-train GNNs, and proposes the graph prompting function to modify the standalone node into a token pair, and reformulate the downstream node classification looking the same as edge prediction.
Abstract: Despite the promising representation learning of graph neural networks (GNNs), the supervised training of GNNs notoriously requires large amounts of labeled data from each application. An effective solution is to apply the transfer learning in graph: using easily accessible information to pre-train GNNs, and fine-tuning them to optimize the downstream task with only a few labels. Recently, many efforts have been paid to design the self-supervised pretext tasks, and encode the universal graph knowledge among the various applications. However, they rarely notice the inherent training objective gap between the pretext and downstream tasks. This significant gap often requires costly fine-tuning for adapting the pre-trained model to downstream problem, which prevents the efficient elicitation of pre-trained knowledge and then results in poor results. Even worse, the naive pre-training strategy usually deteriorates the downstream task, and damages the reliability of transfer learning in graph data. To bridge the task gap, we propose a novel transfer learning paradigm to generalize GNNs, namely graph pre-training and prompt tuning (GPPT). Specifically, we first adopt the masked edge prediction, the most simplest and popular pretext task, to pre-train GNNs. Based on the pre-trained model, we propose the graph prompting function to modify the standalone node into a token pair, and reformulate the downstream node classification looking the same as edge prediction. The token pair is consisted of candidate label class and node entity. Therefore, the pre-trained GNNs could be applied without tedious fine-tuning to evaluate the linking probability of token pair, and produce the node classification decision. The extensive experiments on eight benchmark datasets demonstrate the superiority of GPPT, delivering an average improvement of 4.29% in few-shot graph analysis and accelerating the model convergence up to 4.32X. The code is available in: https://github.com/MingChen-Sun/GPPT.

20 citations

Journal ArticleDOI
TL;DR: In this article , a graph harmonic neural network (GHNN) is proposed to combine the advantages of both graph convolutional networks and graph kernels to leverage the unlabeled data, and thus overcomes label scarcity in semi-supervised scenarios.

14 citations