Spectral clustering and the high-dimensional stochastic blockmodel
TLDR
In this article, the authors studied spectral clustering under the stochastic block model and showed that the eigenvectors of the normalized graph Laplacian asymptotically converges to the eigens of a population normalized graph.Abstract:
Networks or graphs can easily represent a diverse set of data sources that are characterized by interacting units or actors. Social ne tworks, representing people who communicate with each other, are one example. Communities or clusters of highly connected actors form an essential feature in the structure of several empirical networks. Spectral clustering is a popular and computationally feasi ble method to discover these communities. The Stochastic Block Model (Holland et al., 1983) is a social network model with well defined communities; each node is a member of one community. For a network generated from the Stochastic Block Model, we bound the number of nodes "misclus- tered" by spectral clustering. The asymptotic results in th is paper are the first clustering results that allow the number of clusters in the model to grow with the number of nodes, hence the name high-dimensional. In order to study spectral clustering under the Stochastic Block Model, we first show that under the more general latent space model, the eigenvectors of the normalized graph Laplacian asymptotically converge to the eigenvectors of a "population" normal- ized graph Laplacian. Aside from the implication for spectral clustering, this provides insight into a graph visualization technique. Our method of studying the eigenvectors of random matrices is original. AMS 2000 subject classifications: Primary 62H30, 62H25; secondary 60B20.read more
Citations
More filters
Journal ArticleDOI
A review of clustering techniques and developments
Amit Saxena,Mukesh Prasad,Akshansh Gupta,Neha Bharill,Om Prakash Patel,Aruna Tiwari,Meng Joo Er,Weiping Ding,Chin-Teng Lin +8 more
TL;DR: The applications of clustering in some fields like image segmentation, object and character recognition and data mining are highlighted and the approaches used in these methods are discussed with their respective states of art and applicability.
Journal ArticleDOI
Exact Recovery in the Stochastic Block Model
TL;DR: An efficient algorithm based on a semidefinite programming relaxation of ML is proposed, which is proved to succeed in recovering the communities close to the threshold, while numerical experiments suggest that it may achieve the threshold.
Journal ArticleDOI
Consistency of spectral clustering in stochastic block models
Jing Lei,Alessandro Rinaldo +1 more
TL;DR: In this article, the performance of spectral clustering for community extraction in stochastic block models is analyzed and a combinatorial bound on the spectrum of binary random matrices, which is sharper than the conventional matrix Bernstein inequality, is established.
Journal ArticleDOI
A useful variant of the Davis--Kahan theorem for statisticians
TL;DR: In this paper, the authors present a variant of the Davis-Kahan theorem that relies only on a population eigenvalue separation condition, making it more natural and convenient for direct application in statistical contexts, and provide an improvement in many cases to the usual bound.
Journal ArticleDOI
Consistency of spectral clustering in stochastic block models
Jing Lei,Alessandro Rinaldo +1 more
TL;DR: It is shown that, under mild conditions, spectral clustering applied to the adjacency matrix of the network can consistently recover hidden communities even when the order of the maximum expected degree is as small as $\log n$ with $n$ the number of nodes.
References
More filters
Journal ArticleDOI
Collective dynamics of small-world networks
TL;DR: Simple models of networks that can be tuned through this middle ground: regular networks ‘rewired’ to introduce increasing amounts of disorder are explored, finding that these systems can be highly clustered, like regular lattices, yet have small characteristic path lengths, like random graphs.
Journal ArticleDOI
Emergence of Scaling in Random Networks
TL;DR: A model based on these two ingredients reproduces the observed stationary scale-free distributions, which indicates that the development of large networks is governed by robust self-organizing phenomena that go beyond the particulars of the individual systems.
Journal ArticleDOI
Statistical mechanics of complex networks
TL;DR: In this paper, a simple model based on the power-law degree distribution of real networks was proposed, which was able to reproduce the power law degree distribution in real networks and to capture the evolution of networks, not just their static topology.
Book
Social Network Analysis: Methods and Applications
TL;DR: This paper presents mathematical representation of social networks in the social and behavioral sciences through the lens of Dyadic and Triadic Interaction Models, which describes the relationships between actor and group measures and the structure of networks.
Journal ArticleDOI
Community structure in social and biological networks
Michelle Girvan,Mark Newman +1 more
TL;DR: This article proposes a method for detecting communities, built around the idea of using centrality indices to find community boundaries, and tests it on computer-generated and real-world graphs whose community structure is already known and finds that the method detects this known structure with high sensitivity and reliability.