scispace - formally typeset
Open AccessJournal ArticleDOI

Spectral clustering and the high-dimensional stochastic blockmodel

Karl Rohe, +2 more
- 01 Aug 2011 - 
- Vol. 39, Iss: 4, pp 1878-1915
TLDR
In this article, the authors studied spectral clustering under the stochastic block model and showed that the eigenvectors of the normalized graph Laplacian asymptotically converges to the eigens of a population normalized graph.
Abstract
Networks or graphs can easily represent a diverse set of data sources that are characterized by interacting units or actors. Social ne tworks, representing people who communicate with each other, are one example. Communities or clusters of highly connected actors form an essential feature in the structure of several empirical networks. Spectral clustering is a popular and computationally feasi ble method to discover these communities. The Stochastic Block Model (Holland et al., 1983) is a social network model with well defined communities; each node is a member of one community. For a network generated from the Stochastic Block Model, we bound the number of nodes "misclus- tered" by spectral clustering. The asymptotic results in th is paper are the first clustering results that allow the number of clusters in the model to grow with the number of nodes, hence the name high-dimensional. In order to study spectral clustering under the Stochastic Block Model, we first show that under the more general latent space model, the eigenvectors of the normalized graph Laplacian asymptotically converge to the eigenvectors of a "population" normal- ized graph Laplacian. Aside from the implication for spectral clustering, this provides insight into a graph visualization technique. Our method of studying the eigenvectors of random matrices is original. AMS 2000 subject classifications: Primary 62H30, 62H25; secondary 60B20.

read more

Content maybe subject to copyright    Report

Citations
More filters
Journal ArticleDOI

A review of clustering techniques and developments

TL;DR: The applications of clustering in some fields like image segmentation, object and character recognition and data mining are highlighted and the approaches used in these methods are discussed with their respective states of art and applicability.
Journal ArticleDOI

Exact Recovery in the Stochastic Block Model

TL;DR: An efficient algorithm based on a semidefinite programming relaxation of ML is proposed, which is proved to succeed in recovering the communities close to the threshold, while numerical experiments suggest that it may achieve the threshold.
Journal ArticleDOI

Consistency of spectral clustering in stochastic block models

TL;DR: In this article, the performance of spectral clustering for community extraction in stochastic block models is analyzed and a combinatorial bound on the spectrum of binary random matrices, which is sharper than the conventional matrix Bernstein inequality, is established.
Journal ArticleDOI

A useful variant of the Davis--Kahan theorem for statisticians

TL;DR: In this paper, the authors present a variant of the Davis-Kahan theorem that relies only on a population eigenvalue separation condition, making it more natural and convenient for direct application in statistical contexts, and provide an improvement in many cases to the usual bound.
Journal ArticleDOI

Consistency of spectral clustering in stochastic block models

TL;DR: It is shown that, under mild conditions, spectral clustering applied to the adjacency matrix of the network can consistently recover hidden communities even when the order of the maximum expected degree is as small as $\log n$ with $n$ the number of nodes.
References
More filters
Journal ArticleDOI

Collective dynamics of small-world networks

TL;DR: Simple models of networks that can be tuned through this middle ground: regular networks ‘rewired’ to introduce increasing amounts of disorder are explored, finding that these systems can be highly clustered, like regular lattices, yet have small characteristic path lengths, like random graphs.
Journal ArticleDOI

Emergence of Scaling in Random Networks

TL;DR: A model based on these two ingredients reproduces the observed stationary scale-free distributions, which indicates that the development of large networks is governed by robust self-organizing phenomena that go beyond the particulars of the individual systems.
Journal ArticleDOI

Statistical mechanics of complex networks

TL;DR: In this paper, a simple model based on the power-law degree distribution of real networks was proposed, which was able to reproduce the power law degree distribution in real networks and to capture the evolution of networks, not just their static topology.
Book

Social Network Analysis: Methods and Applications

TL;DR: This paper presents mathematical representation of social networks in the social and behavioral sciences through the lens of Dyadic and Triadic Interaction Models, which describes the relationships between actor and group measures and the structure of networks.
Journal ArticleDOI

Community structure in social and biological networks

TL;DR: This article proposes a method for detecting communities, built around the idea of using centrality indices to find community boundaries, and tests it on computer-generated and real-world graphs whose community structure is already known and finds that the method detects this known structure with high sensitivity and reliability.
Related Papers (5)