/pdf/dynamic-network-embedding-survey-1ys7yu1b.pdf

Dynamic network embedding survey

Dynamic Network Embedding Survey

Skeleton-based human action recognition has caused wide concern, as skeleton data can robustly adapt to dynamic circumstances such as camera view changes and background interference thus allowing recognition methods to focus on robust features. In recent studies, the human body is modeled as a topological graph, and the graph convolution network (GCN) is used to extract features of actions. Although GCN has a strong ability to learn spatial modes, it ignores the varying degrees of higher-order dependencies that are captured by message passing. Moreover, the joints represented by vertices are interdependent, and hence incorporating an attention mechanism to weigh dependencies is beneficial. In this work, we propose a kernel attention adaptive graph transformer network (KA-AGTN), which models the higher-order spatial dependencies between joints by the graph transformer operator based on multihead self-attention. In addition, the Temporal Kernel Attention (TKA) block in KA-AGTN generates a channel-level attention score using temporal features, which can enhance temporal motion correlation. After combining the two-stream framework and adaptive graph strategy, KA-AGTN outperforms the baseline 2s-AGCN by 1.9% and by 1% under X-Sub and X-View on the NTU-RGBD 60 dataset, by 3.2% and 3.1% under X-Sub and X-Set on the NTU-RGBD 120 dataset, and by 2% and 2.3% under Top-1 and Top-5 and achieves the state-of-the-art performance on the Kinetics-Skeleton 400 dataset. 

Graph transformer network with temporal kernel attention for skeleton-based action recognition

“Knowledge tracing (KT)” is an emerging and popular research topic in the field of online education that seeks to assess students’ mastery of a concept based on their historical learning of relevant exercises on an online education system in order to make the most accurate prediction of student performance. Since there have been a large number of KT models, we attempt to systematically investigate, compare and discuss different aspects of KT models to find out the differences between these models in order to better assist researchers in this field. The findings of this study have made substantial contributions to the progress of online education, which is especially relevant in light of the current global pandemic. As a result of the current expansion of deep learning methods over the last decade, researchers have been tempted to include deep learning strategies into KT research with astounding results. In this paper, we evaluate current research on deep learning-based KT in the main categories listed below. In particular, we explore (1) a granular categorisation of the technological solutions presented by the mainstream Deep Learning-based KT Models. (2) a detailed analysis of techniques to KT, with a special emphasis on Deep Learning-based KT Models. (3) an analysis of the technological solutions and major improvement presented by Deep Learning-based KT models. In conclusion, we discuss possible future research directions in the field of Deep Learning-based KT. 

A survey on deep learning based knowledge tracing

Aggregation functions are regarded as the multiplication between an aggregation matrix and node embeddings, based on which a full rank matrix can enhance representation capacity of Graph Neural Networks (GNNs). In this work, we fill this research gap based on the full rank aggregation matrix and its functional form, i.e. , the injective aggregation function, and state that injectivity is necessary to guarantee the rich representation capacity to GNNs. To this end, we conduct theoretical injectivity analysis for the typical feature aggregation methods and provide inspiring solutions on turning the non-injective aggregation functions into injective versions. Based on our injective aggregation functions, we create various GNN structures by combining the aggregation functions with the other ingredient of GNNs, node feature encoding, in different ways. Following these structures, we highlight that by using our injective aggregation function entirely as a pre-processing step before applying independent node feature learning, we can simultaneously achieve satisfactory performance and computational efficiency on the large-scale graph-based traffic data for traffic state prediction tasks. Through comprehensive experiments on standard node classification benchmarks and practical traffic state data (for Chengdu and Xi’an cities), we show that the representation capacity of GNNs can be improved by using our injective aggregation functions just by changing the model in simple operations. • Theoretical and practical guidance on deriving injective aggregations from graphs. • Systematical comparison between the injective aggregations. • Advanced efficiency of combination of pre-aggregation and post-encoding. 

Improving performance and efficiency of Graph Neural Networks by injective aggregation

Attributed networks are ubiquitous in the real world, such as social networks. Therefore, many researchers take the node attributes into consideration in the network representation learning to improve the downstream task performance. In this article, we mainly focus on an untouched ``oversmoothing'' problem in the research of the attributed network representation learning. Although the Laplacian smoothing has been applied by the state-of-the-art works to learn a more robust node representation, these works cannot adapt to the topological characteristics of different networks, thereby causing the new oversmoothing problem and reducing the performance on some networks. In contrast, we adopt a smoothing parameter that is evaluated from the topological characteristics of a specified network, such as small worldness or node convergency and, thus, can smooth the nodes' attribute and structure information adaptively and derive both robust and distinguishable node features for different networks. Moreover, we develop an integrated autoencoder to learn the node representation by reconstructing the combination of the smoothed structure and attribute information. By observation of extensive experiments, our approach can preserve the intrinsical information of networks more effectively than the state-of-the-art works on a number of benchmark datasets with very different topological characteristics.

Effective Deep Attributed Network Representation Learning With Topology Adapted Smoothing.

Since many real world networks are evolving over time, such as social networks and user-item networks, there are increasing research efforts on dynamic network embedding in recent years They learn node representations from a sequence of evolving graphs but not only the latest network, for preserving both structural and temporal information from the dynamic networks Due to the lack of comprehensive investigation of them, we give a survey of dynamic network embedding in this paper Our survey inspects the data model, representation learning technique, evaluation and application of current related works and derives common patterns from them Specifically, we present two basic data models, namely, discrete model and continuous model for dynamic networks Correspondingly, we summarize two major categories of dynamic network embedding techniques, namely, structural-first and temporal-first that are adopted by most related works Then we build a taxonomy that refines the category hierarchy by typical learning models The popular experimental data sets and applications are also summarized Lastly, we have a discussion of several distinct research topics in dynamic network embedding

Multimodal datasets represented as tensors oftentimes share some of their modes. However, even though there may exist a one-to-one (or perhaps partial) correspondence between the coupled modes, such correspondence/alignment may not be given, especially when integrating datasets from disparate sources. This is a very important problem, broadly termed as entity alignment or matching, and subsets of the problem such as graph matching have been extremely popular in the recent years. In order to solve this problem, current work computes the alignment based on existing embeddings of the data. This can be problematic if our end goal is the joint analysis of the two datasets into the same latent factor space: the embeddings computed separately per dataset may yield a suboptimal alignment, and if such an alignment is used to subsequently compute the joint latent factors, the computation will similarly be plagued by compounding errors incurred by the imperfect alignment. In this work, we are the first to define and solve the problem of joint tensor alignment and factorization into a shared latent space. By posing this as a unified problem and solving for both tasks simultaneously, we observe that the both alignment and factorization tasks benefit each other resulting in superior performance compared to two-stage approaches. We extensively evaluate our proposed method TENALIGN and conduct a thorough sensitivity and ablation analysis. We demonstrate that TENALIGN significantly outperforms baseline approaches where embedding and matching happen separately.

TENALIGN: Joint Tensor Alignment and Coupled Factorization

—In this paper, we propose a comprehensive unsu- pervised framework that leverages existing and novel multiview learning models, towards obtaining a single node embedding from a collection of node embeddings, combining the best of all worlds. Through extensive experiments, we demonstrate that the proposed multiview node embedding is able to perform on par or better than the best of its constituents and provide reliable performance across downstream tasks including node classification and graph reconstruction.

Jia Chen

Papers

Dynamic Network Embedding Survey

Effective Deep Attributed Network Representation Learning With Topology Adapted Smoothing.

Dynamic Network Embedding Survey

TENALIGN: Joint Tensor Alignment and Coupled Factorization

Unsupervised Multiview Embedding of Node Embeddings