Network embedding assigns nodes in a network to low-dimensional representations and effectively preserves the network structure. Recently, a significant amount of progresses have been made toward this emerging network analysis paradigm. In this survey, we focus on categorizing and then reviewing the current development on network embedding methods, and point out its future research directions. We first summarize the motivation of network embedding. We discuss the classical graph embedding algorithms and their relationship with network embedding. Afterwards and primarily, we provide a comprehensive overview of a large number of network embedding methods in a systematic manner, covering the structure- and property-preserving network embedding methods, the network embedding methods with side information, and the advanced information preserving network embedding methods. Moreover, several evaluation approaches for network embedding and some useful online resources, including the network data sets and softwares, are reviewed, too. Finally, we discuss the framework of exploiting these network embedding methods to build an effective system and point out some potential future directions.

A Survey on Network Embedding

Network embedding, aiming to learn the low-dimensional representations of nodes in networks, is of paramount importance in many real applications. One basic requirement of network embedding is to preserve the structure and inherent properties of the networks. While previous network embedding methods primarily preserve the microscopic structure, such as the first- and second-order proximities of nodes, the mesoscopic community structure, which is one of the most prominent feature of networks, is largely ignored. In this paper, we propose a novel Modularized Nonnegative Matrix Factorization (M-NMF) model to incorporate the community structure into network embedding. We exploit the consensus relationship between the representations of nodes and community structure, and then jointly optimize NMF based representation learning model and modularity based community detection model in a unified framework, which enables the learned representations of nodes to preserve both of the microscopic and community structures. We also provide efficient updating rules to infer the parameters of our model, together with the correctness and convergence guarantees. Extensive experimental results on a variety of real-world networks show the superior performance of the proposed method over the state-of-the-arts.

https://www.aaai.org/ocs/index.php/AAAI/AAAI17/paper/download/14589/13763

Community preserving network embedding

The k-means algorithm is generally the most known and used clustering method. There are various extensions of k-means to be proposed in the literature. Although it is an unsupervised learning to clustering in pattern recognition and machine learning, the k-means algorithm and its extensions are always influenced by initializations with a necessary number of clusters a priori. That is, the k-means algorithm is not exactly an unsupervised clustering method. In this paper, we construct an unsupervised learning schema for the k-means algorithm so that it is free of initializations without parameter selection and can also simultaneously find an optimal number of clusters. That is, we propose a novel unsupervised k-means (U-k-means) clustering algorithm with automatically finding an optimal number of clusters without giving any initialization and parameter selection. The computational complexity of the proposed U-k-means clustering algorithm is also analyzed. Comparisons between the proposed U-k-means and other existing methods are made. Experimental results and comparisons actually demonstrate these good aspects of the proposed U-k-means clustering algorithm.

/pdf/unsupervised-k-means-clustering-algorithm-4ob53iigj3.pdf

Unsupervised K-Means Clustering Algorithm

Domain transfer learning, which learns a target classifier using labeled data from a different distribution, has shown promising value in knowledge discovery yet still been a challenging problem. Most previous works designed adaptive classifiers by exploring two learning strategies independently: distribution adaptation and label propagation. In this paper, we propose a novel transfer learning framework, referred to as Adaptation Regularization based Transfer Learning (ARTL), to model them in a unified way based on the structural risk minimization principle and the regularization theory. Specifically, ARTL learns the adaptive classifier by simultaneously optimizing the structural risk functional, the joint distribution matching between domains, and the manifold consistency underlying marginal distribution. Based on the framework, we propose two novel methods using Regularized Least Squares (RLS) and Support Vector Machines (SVMs), respectively, and use the Representer theorem in reproducing kernel Hilbert space to derive corresponding solutions. Comprehensive experiments verify that ARTL can significantly outperform state-of-the-art learning methods on several public text and image datasets.

Adaptation Regularization: A General Framework for Transfer Learning

Nonnegative matrix factorization (NMF) provides a lower rank approximation of a nonnegative matrix, and has been successfully used as a clustering method. In this paper, we offer some conceptual understanding for the capabilities and shortcomings of NMF as a clustering method. Then, we propose Symmetric NMF (SymNMF) as a general framework for graph clustering, which inherits the advantages of NMF by enforcing nonnegativity on the clustering assignment matrix. Unlike NMF, however, SymNMF is based on a similarity measure between data points, and factorizes a symmetric matrix containing pairwise similarity values (not necessarily nonnegative). We compare SymNMF with the widely-used spectral clustering methods, and give an intuitive explanation of why SymNMF captures the cluster structure embedded in the graph representation more naturally. In addition, we develop a Newton-like algorithm that exploits second-order information efficiently, so as to show the feasibility of SymNMF as a practical framework for graph clustering. Our experiments on artificial graph data, text data, and image data demonstrate the substantially enhanced clustering quality of SymNMF over spectral clustering and NMF. Therefore, SymNMF is able to achieve better clustering results on both linear and nonlinear manifolds, and serves as a potential basis for many extensions

https://www.cc.gatech.edu/~hpark/papers/DaDingParkSDM12.pdf

Symmetric Nonnegative Matrix Factorization for Graph Clustering.

Recently, multiple graph regularizer based methods have shown promising performances in data representation However, the parameter choice of the regularizer is crucial to the performance of clustering and its optimal value changes for different real datasets To deal with this problem, we propose a novel method called Parameter-less Auto-weighted Multiple Graph regularized Nonnegative Matrix Factorization (PAMGNMF) in this paper PAMGNMF employs the linear combination of multiple simple graphs to approximate the manifold structure of data as previous methods do Moreover, the proposed method can automatically learn an optimal weight for each graph without introducing an additive parameter Therefore, the proposed PAMGNMF method is easily applied to practical problems Extensive experimental results on different real-world datasets have demonstrated that the proposed method achieves better performance than the state-of-the-art approaches

Parameter-less Auto-weighted multiple graph regularized Nonnegative Matrix Factorization for data representation

Rank-constrained nonnegative matrix factorization for data representation

Many traditional concept factorization methods employ single graph to approximate the manifold structure of data. Therefore, they cannot capture the underlying geometric structure hidden in data effectively. In this paper, we propose a novel method, called Multiple graph regularized Concept Factorization with Adaptive Weights (MCFAWs), for data representation. It exploits the intrinsic geometric manifold of the data by the linear combination of multiple graphs with parameter free. Therefore, our proposed MCFAW method can be applied to many real problems. Besides, an efficient optimization algorithm is presented to solve the proposed model. Some experimental results on the benchmarks show that the proposed MCFAW method outperforms the state-of-the-art methods.

Multiple Graph Regularized Concept Factorization With Adaptive Weights

Recently, low-rank representation (LRR)-based techniques have manifested remarkable results for data representation. To exploit the latent manifold structure of data, the graph regulariser is incorporated into the model of LRR. However, it is critical to construct an appropriate graph model and set the corresponding parameters. In addition, this procedure is usually time-consuming and proved to be overfitting when using cross validation or discrete grid search. Two novel LRR-based methods, called multiple graph regularised LRR and multiple hypergraph regularised LLR, are proposed to represent the high-dimensional data. To guarantee the smoothness along the estimated manifold, the multiple graph regulariser and the multiple hypergraph regulariser are incorporated into the traditional LRR method, respectively, which results in a unified framework. Moreover, the augmented Lagrange multiplier is adopted to solve the proposed models. Extensive experiments on real image datasets show the effectiveness of the proposed methods.

https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/iet-ipr.2016.0391

Multiple Laplacian graph regularised low-rank representation with application to image representation

Many researchers have paid more attention to the application of non-negative matrix factorization (NMF) in data representation. Recently, some regularization methods can improve the performances by utilizing the data and feature manifold, simultaneously. In this work, a new method, named dual graph regularized NMF with Sinkhorn distance (DSDNMF) is presented. It not only synchronously takes the data structure and feature structure into consideration, but also measures the reconstruction error by adopting the Earth Mover's Distance (EMD) to make full use of the feature correlation. Therefore, DSDNMF can effectively explore the semantic structure information of data in contrast to traditional methods. Besides, we introduce an efficient strategy to optimize our proposed model. Comprehensive experiments on the COIL20 and PIE datasets manifest the superiority of DSDNMF.

Feiyue Ye

Papers

Parameter-less Auto-weighted multiple graph regularized Nonnegative Matrix Factorization for data representation

Rank-constrained nonnegative matrix factorization for data representation

Multiple Graph Regularized Concept Factorization With Adaptive Weights

Multiple Laplacian graph regularised low-rank representation with application to image representation

Dual Graph regularized NMF with Sinkhorn Distance