In recent years, spectral clustering has become one of the most popular modern clustering algorithms. It is simple to implement, can be solved efficiently by standard linear algebra software, and very often outperforms traditional clustering algorithms such as the k-means algorithm. On the first glance spectral clustering appears slightly mysterious, and it is not obvious to see why it works at all and what it really does. The goal of this tutorial is to give some intuition on those questions. We describe different graph Laplacians and their basic properties, present the most common spectral clustering algorithms, and derive those algorithms from scratch by several different approaches. Advantages and disadvantages of the different spectral clustering algorithms are discussed.

/pdf/a-tutorial-on-spectral-clustering-536pmg9onu.pdf

A tutorial on spectral clustering

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

Statistics for Spatial Data.

Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

We propose a framework for analyzing and comparing distributions, which we use to construct statistical tests to determine if two samples are drawn from different distributions. Our test statistic is the largest difference in expectations over functions in the unit ball of a reproducing kernel Hilbert space (RKHS), and is called the maximum mean discrepancy (MMD).We present two distribution free tests based on large deviation bounds for the MMD, and a third test based on the asymptotic distribution of this statistic. The MMD can be computed in quadratic time, although efficient linear time approximations are available. Our statistic is an instance of an integral probability metric, and various classical metrics on distributions are obtained when alternative function classes are used in place of an RKHS. We apply our two-sample tests to a variety of problems, including attribute matching for databases using the Hungarian marriage method, where they perform strongly. Excellent performance is also obtained when comparing distributions over graphs, for which these are the first such tests.

/pdf/a-kernel-two-sample-test-2mr8hwo4ml.pdf

A kernel two-sample test

In Bhatt and Roy's minimal directed spanning tree construction for a random, partially ordered set of points in the unit square, all edges must respect the `coordinatewise' partial order and there must be a directed path from each vertex to a minimal element. We study the asymptotic behaviour of the total length of this graph with power-weighted edges. The limiting distribution is given by the sum of a normal component away from the boundary plus a contribution introduced by the boundary effects, which can be characterized by a fixed-point equation, and is reminiscent of limits arising in the probabilistic analysis of certain algorithms. As the exponent of the power weighting increases, the distribution undergoes a phase transition from the normal contribution being dominant to the boundary effects being dominant. In the critical case in which the weight is simple Euclidean length, both effects contribute significantly to the limit law.

/pdf/on-the-total-length-of-the-random-minimal-directed-spanning-45363khpvi.pdf

On the total length of the random minimal directed spanning tree.

Suppose $X = (X_x, x$ in $Z^d)$ is a family of i.i.d. variables in some measurable space, $B_0$ is a bounded set in $R^d$, and for $t > 1$, $H_t$ is a measure on $tB_0$ determined by the restriction of $X$ to lattice sites in or adjacent to $tB_0$. We prove convergence to a white noise process for the random measure on $B_0$ given by $t^{-d/2}(H_t(tA)-EH_t(tA))$ for subsets $A$ of $B_0$, as $t$ becomes large,subject to $H$ satisfying a ``stabilization'' condition (whereby the effect of changing $X$ at a single site $x$ is local) but with no assumptions on the rate of decay of correlations. We also give a multivariate central limit theorem for the joint distributions of two or more such measures $H_t$, and adapt the result to measures based on Poisson and binomial point processes. Applications given include a white noise limit for the measure which counts clusters of critical percolation, a functional central limit theorem for the empirical process of the edge lengths of the minimal spanning tree on random points, and central limit theorems for the on-line nearest neighbour graph.

Multivariate spatial central limit theorems with applications to percolation and spatial graphs

This work deals with convergence theorems and bounds on the cost of several layout measures for lattice graphs, random lattice graphs and sparse random geometric graphs. Specifically, we consider the following problems: Minimum Linear Arrangement, Cutwidth, Sum Cut, Vertex Separation, Edge Bisection and Vertex Bisection. For full square lattices, we give optimal layouts for the problems still open. For arbitrary lattice graphs, we present best possible bounds disregarding a constant factor. We apply percolation theory to the study of lattice graphs in a probabilistic setting. In particular, we deal with the subcritical regime that this class of graphs exhibits and characterize the behaviour of several layout measures in this space of probability. We extend the results on random lattice graphs to random geometric graphs, which are graphs whose nodes are spread at random in the unit square and whose edges connect pairs of points which are within a given distance. We also characterize the behaviour of several layout measures on random geometric graphs in their subcritical regime. Our main results are convergence theorems that can be viewed as an analogue of the Beardwood, Halton and Hammersley theorem for the Euclidean TSP on random points in the unit square.

/pdf/convergence-theorems-for-some-layout-measures-on-random-3vicfqb2rp.pdf

Convergence Theorems for Some Layout Measures on Random Lattice and Random Geometric Graphs

Single linkage clustering and continuum percolation

Nearest neighbor cells in Rd, d∈ℕ, are used to define coefficients of divergence (φ-divergences) between continuous multivariate samples. For large sample sizes, such distances are shown to be asymptotically normal with a variance depending on the underlying point density. In d=1, this extends classical central limit theory for sum functions of spacings. The general results yield central limit theorems for logarithmic k-spacings, information gain, log-likelihood ratios and the number of pairs of sample points within a fixed distance of each other.

/pdf/gaussian-limits-for-generalized-spacings-3oixpwq1r0.pdf

Mathew D. Penrose

Papers

On the total length of the random minimal directed spanning tree.

Multivariate spatial central limit theorems with applications to percolation and spatial graphs

Convergence Theorems for Some Layout Measures on Random Lattice and Random Geometric Graphs

Single linkage clustering and continuum percolation

Gaussian limits for generalized spacings.