In recent years, spectral clustering has become one of the most popular modern clustering algorithms. It is simple to implement, can be solved efficiently by standard linear algebra software, and very often outperforms traditional clustering algorithms such as the k-means algorithm. On the first glance spectral clustering appears slightly mysterious, and it is not obvious to see why it works at all and what it really does. The goal of this tutorial is to give some intuition on those questions. We describe different graph Laplacians and their basic properties, present the most common spectral clustering algorithms, and derive those algorithms from scratch by several different approaches. Advantages and disadvantages of the different spectral clustering algorithms are discussed.

/pdf/a-tutorial-on-spectral-clustering-536pmg9onu.pdf

A tutorial on spectral clustering

We will review some of the major results in random graphs and some of the more challenging open problems. We will cover algorithmic and structural questions. We will touch on newer models, including those related to the WWW.

Random graphs

Statistics for Spatial Data.

Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

We propose a framework for analyzing and comparing distributions, which we use to construct statistical tests to determine if two samples are drawn from different distributions. Our test statistic is the largest difference in expectations over functions in the unit ball of a reproducing kernel Hilbert space (RKHS), and is called the maximum mean discrepancy (MMD).We present two distribution free tests based on large deviation bounds for the MMD, and a third test based on the asymptotic distribution of this statistic. The MMD can be computed in quadratic time, although efficient linear time approximations are available. Our statistic is an instance of an integral probability metric, and various classical metrics on distributions are obtained when alternative function classes are used in place of an RKHS. We apply our two-sample tests to a variety of problems, including attribute matching for databases using the Hungarian marriage method, where they perform strongly. Excellent performance is also obtained when comparing distributions over graphs, for which these are the first such tests.

/pdf/a-kernel-two-sample-test-2mr8hwo4ml.pdf

A kernel two-sample test

Consider throwing $n$ balls at random into $m$ urns, each ball landing in urn $i$ with probability $p_i$. Let $S$ be the resulting number of singletons, i.e., urns containing just one ball. We give an error bound for the Kolmogorov distance from $S$ to the normal, and estimates on its variance. These show that if $n$, $m$ and $(p_i, 1 \leq i \leq m)$ vary in such a way that $\sup_i p_i = O(n^{-1})$, then $S$ satisfies a CLT if and only if $n^2 \sum_i p_i^2$ tends to infinity, and demonstrate an optimal rate of convergence in the CLT in this case. In the uniform case $(p_i \equiv m^{-1}) with $m$ and $n$ growing proportionately, we provide bounds with better asymptotic constants. The proof of the error bounds are based on Stein's method via size-biased couplings.

Normal approximation for isolated balls in an urn allocation model

For a 3-dimensional system of hard spheres of diameter D and mass m with an added attractive square-well two-body interaction of width a and depth e, let BD, a denote the quantum second virial coefficient. Let BD denote the quantum second virial coefficient for hard spheres of diameter D without the added attractive interaction. We show that in the limit a → 0 at constant α: = ℰma2/(2ħ2) with α π2/8, we argue that the gaseous phase disappears in the limit a → 0, so that the second virial coefficient becomes irrelevant.

Sticky Spheres in Quantum Mechanics

Let $d \geq 2$. The Cheeger constant of a graph is the minimum surface-to-volume ratio of all subsets of the vertex set with relative volume at most 1/2. There are several ways to define surface and volume here: the simplest method is to count boundary edges (for the surface) and vertices (for the volume). We show that for a geometric (possibly weighted) graph on $n$ random points in a $d$-dimensional domain with Lipschitz boundary and with distance parameter decaying more slowly (as a function of $n$) than the connectivity threshold, the Cheeger constant (under several possible definitions of surface and volume), also known as conductance, suitably rescaled, converges for large $n$ to an analogous Cheeger-type constant of the domain. Previously, Garc\'ia Trillos {\em et al.} had shown this for $d \geq 3$ but had required an extra condition on the distance parameter when $d=2$.

Optimal Cheeger cuts and bisections of random geometric graphs

We consider a Poisson process $\eta$ on an arbitrary measurable space with an arbitrary sigma-finite intensity measure. We establish an explicit Fock space representation of square integrable functions of $\eta$. As a consequence we identify explicitly, in terms of iterated difference operators, the integrands in the Wiener-Ito chaos expansion. We apply these results to extend well-known variance inequalities for homogeneous Poisson processes on the line to the general Poisson case. The Poincare inequality is a special case. Further applications are covariance identities for Poisson processes on (strictly) ordered spaces and Harris-FKG-inequalities for monotone functions of $\eta$.

Mathew D. Penrose

Papers

Normal approximation for isolated balls in an urn allocation model

Sticky Spheres in Quantum Mechanics

Optimal Cheeger cuts and bisections of random geometric graphs

Poisson process Fock space representation, chaos expansion and covariance inequalities

Lectures on random geometric graphs