Spectral Properties of the Alignment Matrices in Manifold Learning

TLDR: It is shown that the gap in the spectrum of the alignment matrix is proportional to the square of the size of the overlap of the local parameterizations, thus deriving a quantitative measure of how stably the null space can be computed numerically.

Abstract: Local methods for manifold learning generate a collection of local parameterizations which is then aligned to produce a global parameterization of the underlying manifold. The alignment procedure is carried out through the computation of a partial eigendecomposition of a so-called alignment matrix. In this paper, we present an analysis of the eigenstructure of the alignment matrix giving both necessary and sufficient conditions under which the null space of the alignment matrix recovers the global parameterization. We show that the gap in the spectrum of the alignment matrix is proportional to the square of the size (the precise definition of the size is given in section sec:sp) of the overlap of the local parameterizations, thus deriving a quantitative measure of how stably the null space can be computed numerically. We also give a perturbation analysis of the null space of the alignment matrix when the computation of the local parameterizations is subject to error. Our analysis provides insights into the behaviors and performance of local manifold learning algorithms.