Community discovery using nonnegative matrix factorization
TL;DR: This paper investigates another important issue, community discovery, in network analysis, and chooses Nonnegative Matrix Factorization (NMF) as a tool to find the communities because of its powerful interpretability and close relationship between clustering methods.
Abstract: Complex networks exist in a wide range of real world systems, such as social networks, technological networks, and biological networks. During the last decades, many researchers have concentrated on exploring some common things contained in those large networks include the small-world property, power-law degree distributions, and network connectivity. In this paper, we will investigate another important issue, community discovery, in network analysis. We choose Nonnegative Matrix Factorization (NMF) as our tool to find the communities because of its powerful interpretability and close relationship between clustering methods. Targeting different types of networks (undirected, directed and compound), we propose three NMF techniques (Symmetric NMF, Asymmetric NMF and Joint NMF). The correctness and convergence properties of those algorithms are also studied. Finally the experiments on real world networks are presented to show the effectiveness of the proposed methods.
Citations
More filters
Proceedings Article•
04 Feb 2017TL;DR: A novel Modularized Nonnegative Matrix Factorization (M-NMF) model is proposed to incorporate the community structure into network embedding and jointly optimize NMF based representation learning model and modularity based community detection model in a unified framework, which enables the learned representations of nodes to preserve both of the microscopic and community structures.
Abstract: Network embedding, aiming to learn the low-dimensional representations of nodes in networks, is of paramount importance in many real applications. One basic requirement of network embedding is to preserve the structure and inherent properties of the networks. While previous network embedding methods primarily preserve the microscopic structure, such as the first- and second-order proximities of nodes, the mesoscopic community structure, which is one of the most prominent feature of networks, is largely ignored. In this paper, we propose a novel Modularized Nonnegative Matrix Factorization (M-NMF) model to incorporate the community structure into network embedding. We exploit the consensus relationship between the representations of nodes and community structure, and then jointly optimize NMF based representation learning model and modularity based community detection model in a unified framework, which enables the learned representations of nodes to preserve both of the microscopic and community structures. We also provide efficient updating rules to infer the parameters of our model, together with the correctness and convergence guarantees. Extensive experimental results on a variety of real-world networks show the superior performance of the proposed method over the state-of-the-arts.
756 citations
Cites background from "Community discovery using nonnegati..."
...(20) By lemma 2 in (Wang et al. 2011), we have −2αtr(HCUT ) ≤ −2αtr(CUTZ)− 2αtr(CUTH′)....
[...]
...(19) By lemma 6 in (Wang et al. 2011), we have βtr(HTB1H) ≤ 1 2 βtr(YTB1H ′) + 1 2 βtr(H′ T B1Y)....
[...]
...(21) By lemmas 6 and 7 in (Wang et al. 2011), we have...
[...]
...By lemma 4 in (Wang et al. 2011), we have −βtr(HTAH) ≤− βtr(H′TAZ)− βtr(ZTAH′) − βtr(H′TAH′), (18) and − (2λ− α)tr(HTH) ≤ −(2λ− α)tr(H′TZ) − (2λ− α)tr(ZTH′)− (2λ− α)tr(H′TH′)....
[...]
...(21) By lemmas 6 and 7 in (Wang et al. 2011), we have λtr(HTHHTH) ≤ λtr(PH′TH′) ≤ λtr(RH′TH′H′T ), (22) where Pij = (UTU)2ij (H′TH′)ij and Rij = H4ij H ′3 ij ....
[...]
TL;DR: This work describes a method for finding overlapping communities based on a principled statistical approach using generative network models and shows how the method can be implemented using a fast, closed-form expectation-maximization algorithm that allows us to analyze networks of millions of nodes in reasonable running times.
Abstract: A fundamental problem in the analysis of network data is the detection of network communities, groups of densely interconnected nodes, which may be overlapping or disjoint. Here we describe a method for finding overlapping communities based on a principled statistical approach using generative network models. We show how the method can be implemented using a fast, closed-form expectation-maximization algorithm that allows us to analyze networks of millions of nodes in reasonable running times. We test the method both on real-world networks and on synthetic benchmarks and find that it gives results competitive with previous methods. We also show that the same approach can be used to extract nonoverlapping community divisions via a relaxation method, and demonstrate that the algorithm is competitively fast and accurate for the nonoverlapping problem.
412 citations
Posted Content•
TL;DR: Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state- of- the-art semi-supervised models.
Abstract: This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the representations of substructures of different scales (e.g., nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures. Furthermore, we further propose InfoGraph*, an extension of InfoGraph for semi-supervised scenarios. InfoGraph* maximizes the mutual information between unsupervised graph representations learned by InfoGraph and the representations learned by existing supervised methods. As a result, the supervised encoder learns from unlabeled data while preserving the latent semantic space favored by the current supervised task. Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state-of-the-art semi-supervised models.
394 citations
Cites background from "Community discovery using nonnegati..."
...There has been a significant amount of previous work done studying many aspects of graphs including link prediction [13, 57] and node prediction [2]....
[...]
Posted Content•
TL;DR: A recent subclass of NMF problems is presented, referred to as near-separable NMF, that can be solved efficiently (that is, in polynomial time), even in the presence of noise.
Abstract: Nonnegative matrix factorization (NMF) has become a widely used tool for the analysis of high-dimensional data as it automatically extracts sparse and meaningful features from a set of nonnegative data vectors. We first illustrate this property of NMF on three applications, in image processing, text mining and hyperspectral imaging --this is the why. Then we address the problem of solving NMF, which is NP-hard in general. We review some standard NMF algorithms, and also present a recent subclass of NMF problems, referred to as near-separable NMF, that can be solved efficiently (that is, in polynomial time), even in the presence of noise --this is the how. Finally, we briefly describe some problems in mathematics and computer science closely related to NMF via the nonnegative rank.
330 citations
Cites background from "Community discovery using nonnegati..."
...Other applications include air emission control [97], computational biology [34], blind source separation [22], single-channel source separation [82], clustering [35], music analysis [42], collaborative filtering [92], and community detection [106]....
[...]
References
More filters
TL;DR: Simple models of networks that can be tuned through this middle ground: regular networks ‘rewired’ to introduce increasing amounts of disorder are explored, finding that these systems can be highly clustered, like regular lattices, yet have small characteristic path lengths, like random graphs.
Abstract: Networks of coupled dynamical systems have been used to model biological oscillators, Josephson junction arrays, excitable media, neural networks, spatial games, genetic control networks and many other self-organizing systems. Ordinarily, the connection topology is assumed to be either completely regular or completely random. But many biological, technological and social networks lie somewhere between these two extremes. Here we explore simple models of networks that can be tuned through this middle ground: regular networks 'rewired' to introduce increasing amounts of disorder. We find that these systems can be highly clustered, like regular lattices, yet have small characteristic path lengths, like random graphs. We call them 'small-world' networks, by analogy with the small-world phenomenon (popularly known as six degrees of separation. The neural network of the worm Caenorhabditis elegans, the power grid of the western United States, and the collaboration graph of film actors are shown to be small-world networks. Models of dynamical systems with small-world coupling display enhanced signal-propagation speed, computational power, and synchronizability. In particular, infectious diseases spread more easily in small-world networks than in regular lattices.
39,297 citations
"Community discovery using nonnegati..." refers background in this paper
...Nowadays, complex networks exist in a wide variety of systems in different areas, such as social networks (Scott 2000; Wasserman and Faust 1994), technological networks (Amaral et al. 2000; Watts and Strogatz 1998 ), biological networks (Sharan 2005; Watts and Strogatz 1998) and information networks (Albert et al. 1999; Faloutsos et al.)....
[...]
...…variety of systems in different areas, such as social networks (Scott 2000; Wasserman and Faust 1994), technological networks (Amaral et al. 2000; Watts and Strogatz 1998), biological networks (Sharan 2005; Watts and Strogatz 1998) and information networks (Albert et al. 1999; Faloutsos et al.)....
[...]
...Nowadays, complex networks exist in a wide variety of systems in different areas, such as social networks (Scott 2000; Wasserman and Faust 1994), technological networks (Amaral et al. 2000; Watts and Strogatz 1998), biological networks (Sharan 2005; Watts and Strogatz 1998 ) and information networks (Albert et al. 1999; Faloutsos et al.)....
[...]
TL;DR: This article proposes a method for detecting communities, built around the idea of using centrality indices to find community boundaries, and tests it on computer-generated and real-world graphs whose community structure is already known and finds that the method detects this known structure with high sensitivity and reliability.
Abstract: A number of recent studies have focused on the statistical properties of networked systems such as social networks and the Worldwide Web. Researchers have concentrated particularly on a few properties that seem to be common to many networks: the small-world property, power-law degree distributions, and network transitivity. In this article, we highlight another property that is found in many networks, the property of community structure, in which network nodes are joined together in tightly knit groups, between which there are only looser connections. We propose a method for detecting such communities, built around the idea of using centrality indices to find community boundaries. We test our method on computer-generated and real-world graphs whose community structure is already known and find that the method detects this known structure with high sensitivity and reliability. We also apply the method to two networks whose community structure is not well known—a collaboration network and a food web—and find that it detects significant and informative community divisions in both cases.
14,429 citations
"Community discovery using nonnegati..." refers background in this paper
...2005), which are usually called clusters or communities (Girvan and Newman 2002)....
[...]
...Besides that, most real world networks demonstrate that the nodes (or units) contained in their certain parts are densely connected to each other (Palla et al. 2005), which are usually called clusters or communities (Girvan and Newman 2002)....
[...]
TL;DR: An algorithm for non-negative matrix factorization is demonstrated that is able to learn parts of faces and semantic features of text and is in contrast to other methods that learn holistic, not parts-based, representations.
Abstract: Is perception of the whole based on perception of its parts? There is psychological and physiological evidence for parts-based representations in the brain, and certain computational theories of object recognition rely on such representations. But little is known about how brains or computers might learn the parts of objects. Here we demonstrate an algorithm for non-negative matrix factorization that is able to learn parts of faces and semantic features of text. This is in contrast to other methods, such as principal components analysis and vector quantization, that learn holistic, not parts-based, representations. Non-negative matrix factorization is distinguished from the other methods by its use of non-negativity constraints. These constraints lead to a parts-based representation because they allow only additive, not subtractive, combinations. When non-negative matrix factorization is implemented as a neural network, parts-based representations emerge by virtue of two properties: the firing rates of neurons are never negative and synaptic strengths do not change sign.
11,500 citations
"Community discovery using nonnegati..." refers methods in this paper
...It was originally proposed as a method for finding matrix factors with parts-of-whole interpretations ( Lee and Seung 1999 )....
[...]
...It was originally proposed as a method for finding matrix factors with parts-of-whole interpretations (Lee and Seung 1999)....
[...]
01 Jan 1999
TL;DR: In this article, non-negative matrix factorization is used to learn parts of faces and semantic features of text, which is in contrast to principal components analysis and vector quantization that learn holistic, not parts-based, representations.
Abstract: Is perception of the whole based on perception of its parts? There is psychological and physiological evidence for parts-based representations in the brain, and certain computational theories of object recognition rely on such representations. But little is known about how brains or computers might learn the parts of objects. Here we demonstrate an algorithm for non-negative matrix factorization that is able to learn parts of faces and semantic features of text. This is in contrast to other methods, such as principal components analysis and vector quantization, that learn holistic, not parts-based, representations. Non-negative matrix factorization is distinguished from the other methods by its use of non-negativity constraints. These constraints lead to a parts-based representation because they allow only additive, not subtractive, combinations. When non-negative matrix factorization is implemented as a neural network, parts-based representations emerge by virtue of two properties: the firing rates of neurons are never negative and synaptic strengths do not change sign.
9,604 citations
TL;DR: In this article, the authors present the most common spectral clustering algorithms, and derive those algorithms from scratch by several different approaches, and discuss the advantages and disadvantages of these algorithms.
Abstract: In recent years, spectral clustering has become one of the most popular modern clustering algorithms. It is simple to implement, can be solved efficiently by standard linear algebra software, and very often outperforms traditional clustering algorithms such as the k-means algorithm. On the first glance spectral clustering appears slightly mysterious, and it is not obvious to see why it works at all and what it really does. The goal of this tutorial is to give some intuition on those questions. We describe different graph Laplacians and their basic properties, present the most common spectral clustering algorithms, and derive those algorithms from scratch by several different approaches. Advantages and disadvantages of the different spectral clustering algorithms are discussed.
9,141 citations