Showing papers on "Spectral graph theory published in 2016"

Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Abstract: In this work, we are interested in generalizing convolutional neural networks (CNNs) from low-dimensional regular grids, where image, video and speech are represented, to high-dimensional irregular domains, such as social networks, brain connectomes or words' embedding, represented by graphs. We present a formulation of CNNs in the context of spectral graph theory, which provides the necessary mathematical background and efficient numerical schemes to design fast localized convolutional filters on graphs. Importantly, the proposed technique offers the same linear computational complexity and constant learning complexity as classical CNNs, while being universal to any graph structure. Experiments on MNIST and 20NEWS demonstrate the ability of this novel deep learning system to learn local, stationary, and compositional features on graphs.

Convolutional neural networks on graphs with fast localized spectral filtering

Abstract: In this work, we are interested in generalizing convolutional neural networks (CNNs) from low-dimensional regular grids, where image, video and speech are represented, to high-dimensional irregular domains, such as social networks, brain connectomes or words' embedding, represented by graphs. We present a formulation of CNNs in the context of spectral graph theory, which provides the necessary mathematical background and efficient numerical schemes to design fast localized convolutional filters on graphs. Importantly, the proposed technique offers the same linear computational complexity and constant learning complexity as classical CNNs, while being universal to any graph structure. Experiments on MNIST and 20NEWS demonstrate the ability of this novel deep learning system to learn local, stationary, and compositional features on graphs.

Spectral-clustering approach to Lagrangian vortex detection.

Abstract: One of the ubiquitous features of real-life turbulent flows is the existence and persistence of coherent vortices. Here we show that such coherent vortices can be extracted as clusters of Lagrangian trajectories. We carry out the clustering on a weighted graph, with the weights measuring pairwise distances of fluid trajectories in the extended phase space of positions and time. We then extract coherent vortices from the graph using tools from spectral graph theory. Our method locates all coherent vortices in the flow simultaneously, thereby showing high potential for automated vortex tracking. We illustrate the performance of this technique by identifying coherent Lagrangian vortices in several two- and three-dimensional flows.

Network Topology Inference from Spectral Templates

Abstract: We address the problem of identifying a graph structure from the observation of signals defined on its nodes. Fundamentally, the unknown graph encodes direct relationships between signal elements, which we aim to recover from observable indirect relationships generated by a diffusion process on the graph. The fresh look advocated here permeates benefits from convex optimization and stationarity of graph signals, in order to identify the graph shift operator (a matrix representation of the graph) given only its eigenvectors. These spectral templates can be obtained, e.g., from the sample covariance of independent graph signals diffused on the sought network. The novel idea is to find a graph shift that, while being consistent with the provided spectral information, endows the network with certain desired properties such as sparsity. To that end we develop efficient inference algorithms stemming from provably-tight convex relaxations of natural nonconvex criteria, particularizing the results for two shifts: the adjacency matrix and the normalized Laplacian. Algorithms and theoretical recovery conditions are developed not only when the templates are perfectly known, but also when the eigenvectors are noisy or when only a subset of them are given. Numerical tests showcase the effectiveness of the proposed algorithms in recovering social, brain, and amino-acid networks.

Generalized Laplacian precision matrix estimation for graph signal processing

Abstract: Graph signal processing models high dimensional data as functions on the vertices of a graph. This theory is constructed upon the interpretation of the eigenvectors of the Laplacian matrix as the Fourier transform for graph signals. We formulate the graph learning problem as a precision matrix estimation with generalized Laplacian constraints, and we propose a new optimization algorithm. Our formulation takes a covariance matrix as input and at each iteration updates one row/column of the precision matrix by solving a non-negative quadratic program. Experiments using synthetic data with generalized Laplacian precision matrix show that our method detects the nonzero entries and it estimates its values more precisely than the graphical Lasso. For texture images we obtain graphs whose edges follow the orientation. We show our graphs are more sparse than the ones obtained using other graph learning methods.

Dual Graph Regularized Dictionary Learning

Abstract: Dictionary learning (DL) techniques aim to find sparse signal representations that capture prominent characteristics in a given data. Such methods operate on a data matrix $Y\in \mathbb {R}^{N\times M}$ , where each of its columns $y_i\in \mathbb {R}^N$ constitutes a training sample, and these columns together represent a sampling from the data manifold. For signals $y\in \mathbb {R}^N$ residing on weighted graphs, an additional challenge is incorporating the underlying geometric structure of the data domain into the learning process. In such cases, the topological graph structure may provide a crucial interpretation for the columns, while the data manifold itself may also possess a low-dimensional intrinsic structure that should be taken into account. In this work, we propose a novel dictionary learning algorithm for graph signals that simultaneously takes into account the underlying structure in both the signal and the manifold domains. Specifically, we require that the dictionary atoms are smooth with respect to the graph topology, as encapsulated by the graph Laplacian matrix. Furthermore, we propose to learn this graph Laplacian within the dictionary learning process, adapting it to promote the desired smoothness. Utilizing the manifold structure, we propose to encourage the smoothness of the sparse representations on the data manifold in a similar manner. Both these smoothness forces implicitly enhance the learned dictionary. The efficiency of the proposed approach is demonstrated on synthetic examples as well as on real data, showing that it outperforms other dictionary learning methods in typical problems such as resistance to noise and data completion.

Characterization and Inference of Graph Diffusion Processes from Observations of Stationary Signals

Abstract: Many tools from the field of graph signal processing exploit knowledge of the underlying graph's structure (e.g., as encoded in the Laplacian matrix) to process signals on the graph. Therefore, in the case when no graph is available, graph signal processing tools cannot be used anymore. Researchers have proposed approaches to infer a graph topology from observations of signals on its nodes. Since the problem is ill-posed, these approaches make assumptions, such as smoothness of the signals on the graph, or sparsity priors. In this paper, we propose a characterization of the space of valid graphs, in the sense that they can explain stationary signals. To simplify the exposition in this paper, we focus here on the case where signals were i.i.d. at some point back in time and were observed after diffusion on a graph. We show that the set of graphs verifying this assumption has a strong connection with the eigenvectors of the covariance matrix, and forms a convex set. Along with a theoretical study in which these eigenvectors are assumed to be known, we consider the practical case when the observations are noisy, and experimentally observe how fast the set of valid graphs converges to the set obtained when the exact eigenvectors are known, as the number of observations grows. To illustrate how this characterization can be used for graph recovery, we present two methods for selecting a particular point in this set under chosen criteria, namely graph simplicity and sparsity. Additionally, we introduce a measure to evaluate how much a graph is adapted to signals under a stationarity assumption. Finally, we evaluate how state-of-the-art methods relate to this framework through experiments on a dataset of temperatures.

Coherent structure coloring: identification of coherent structures from sparse data using graph theory

Abstract: We present a frame-invariant method for detecting coherent structures from Lagrangian flow trajectories that can be sparse in number, as is the case in many fluid mechanics applications of practical interest. The method, based on principles used in graph coloring and spectral graph drawing algorithms, examines a measure of the kinematic dissimilarity of all pairs of fluid trajectories, either measured experimentally, e.g. using particle tracking velocimetry; or numerically, by advecting fluid particles in the Eulerian velocity field. Coherence is assigned to groups of particles whose kinematics remain similar throughout the time interval for which trajectory data is available, regardless of their physical proximity to one another. Through the use of several analytical and experimental validation cases, this algorithm is shown to robustly detect coherent structures using significantly less flow data than is required by existing spectral graph theory methods.

On the spectral gap of a quantum graph

Abstract: We consider the problem of finding universal bounds of “isoperimetric” or “isodiametric” type on the spectral gap of the Laplacian on a metric graph with natural boundary conditions at the vertices, in terms of various analytical and combinatorial properties of the graph: its total length, diameter, number of vertices and number of edges. We investigate which combinations of parameters are necessary to obtain non-trivial upper and lower bounds and obtain a number of sharp estimates in terms of these parameters. We also show that, in contrast to the Laplacian matrix on a combinatorial graph, no bound depending only on the diameter is possible. As a special case of our results on metric graphs, we deduce estimates for the normalised Laplacian matrix on combinatorial graphs which, surprisingly, are sometimes sharper than the ones obtained by purely combinatorial methods in the graph theoretical literature.

Spectrum of the 1-Laplacian and Cheeger's Constant on Graphs

Abstract: We develop a nonlinear spectral graph theory, in which the Laplace operator is replaced by the 1 - Laplacian Δ1. The eigenvalue problem is to solve a nonlinear system involving a set valued function. In the study, we investigate the structure of the solutions, the minimax characterization of eigenvalues, the multiplicity theorem, etc. The eigenvalues as well as the eigenvectors are computed for several elementary graphs. The graphic feature of eigenvalues are also studied. In particular, Cheeger's constant, which has only some upper and lower bounds in linear spectral theory, equals to the first nonzero Δ1 eigenvalue for connected graphs.

Graph connection Laplacian methods can be made robust to noise

Abstract: Recently, several data analytic techniques based on graph connection Laplacian (GCL) ideas have appeared in the literature. At this point, the properties of these methods are starting to be understood in the setting where the data is observed without noise. We study the impact of additive noise on these methods and show that they are remarkably robust. As a by-product of our analysis, we propose modifications of the standard algorithms that increase their robustness to noise. We illustrate our results in numerical simulations.

Eigenvectors of random matrices: A survey

Abstract: Eigenvectors of large matrices (and graphs) play an essential role in combinatorics and theoretical computer science. The goal of this survey is to provide an up-to-date account on properties of eigenvectors when the matrix (or graph) is random.

Data-Driven Partitioning of Power Networks Via Koopman Mode Analysis

Abstract: This paper applies a new technique for modal decomposition based solely on measurements to test systems and demonstrates the technique's capability for partitioning a power network, which determines the points of separation in an islanding strategy. The mathematical technique is called the Koopman mode analysis (KMA) and stems from a spectral analysis of the so-called Koopman operator. Here, KMA is numerically approximated by applying an Arnoldi-like algorithm recently first applied to power system dynamics. In this paper we propose a practical data-driven algorithm incorporating KMA for network partitioning. Comparisons are made with two techniques previously applied for the network partitioning: spectral graph theory which is based on the eigenstructure of the graph Laplacian, and slow-coherency which identifies coherent groups of generators for a specified number of low-frequency modes. The partitioning results share common features with results obtained with graph theory and slow-coherency-based techniques. The suggested partitioning method is evaluated with two test systems, and similarities between Koopman modes and Laplacian eigenvectors are showed numerically and elaborated theoretically.

Detecting Anomalous Activity on Networks With the Graph Fourier Scan Statistic

Abstract: We consider the problem of deciding, based on a single noisy measurement at each vertex of a given graph, whether the underlying unknown signal is constant over the graph or there exists a cluster of vertices with anomalous activation. This problem is relevant to several applications such as surveillance, disease outbreak detection, biomedical imaging, environmental monitoring, etc. Since the activations in these problems often tend to be localized to small groups of vertices in the graphs, we model such activity by a class of signals that are elevated over a (possibly disconnected) cluster with low cut size relative to its size. We analyze the corresponding generalized likelihood ratio (GLR) statistics and relate it to the problem of finding a sparsest cut in the graph. We develop a convex relaxation of the GLR statistic based on spectral graph theory, which we call the graph Fourier scan statistic (GFSS). In our main theoretical result, we show that the performance of the GFSS depends explicitly on the spectral properties of the graph. To assess the optimality of the GFSS, we prove an information theoretic lower bound for the detection of anomalous activity on graphs. Because the GFSS requires the specification of a tuning parameter, we develop an adaptive version of the GFSS. Using these results, we are able to characterize in a very explicit form the performance of the GFSS on a few notable graph topologies. We demonstrate that the GFSS can efficiently detect a simulated Arsenic contamination in groundwater.

Algebraic aspects of the normalized Laplacian

Abstract: Spectral graph theory looks at the interplay between the structure of a graph and the eigenvalues of a matrix associated with the graph. Many interesting graphs have rich structure which can help in determining the eigenvalues associated with a particular graph matrix. This survey looks at some common techniques in working with and determining the eigenvalues associated with the normalized Laplacian matrix, in addition to some algebraic applications of these eigenvalues.

Graph Laplacian Regularization for Inverse Imaging: Analysis in the Continuous Domain.

Abstract: Inverse imaging problems are inherently under-determined, and hence it is important to employ appropriate image priors for regularization. One recent popular prior---the graph Laplacian regularizer---assumes that the target pixel patch is smooth with respect to an appropriately chosen graph. However, the mechanisms and implications of imposing the graph Laplacian regularizer on the original inverse problem are not well understood. To address this problem, in this paper we interpret neighborhood graphs of pixel patches as discrete counterparts of Riemannian manifolds and perform analysis in the continuous domain, providing insights into several fundamental aspects of graph Laplacian regularization. Specifically, we first show the convergence of the graph Laplacian regularizer to a continuous-domain functional, integrating a norm measured in a locally adaptive metric space. Focusing on image denoising, we derive an optimal metric space assuming nonlocal self-similarity of pixel patches, leading to an optimal graph Laplacian regularizer for denoising in the discrete domain. We then interpret graph Laplacian regularization as an anisotropic diffusion scheme to explain its behavior during iterations, e.g., its tendency to promote piecewise smooth signals under certain settings. To verify our analysis, an iterative image denoising algorithm is developed. Experimental results show that our algorithm performs competitively with state-of-the-art denoising methods such as BM3D for natural images, and outperforms them significantly for piecewise smooth images.

Faster Algorithms for Computing the Stationary Distribution, Simulating Random Walks, and More

Abstract: In this paper, we provide faster algorithms for computing various fundamental quantities associated with random walks on a directed graph, including the stationary distribution, personalized PageRank vectors, hitting times, and escape probabilities. In particular, on a directed graph with $n$ vertices and $m$ edges, we show how to compute each quantity in time $\tilde{O}(m^{3/4}n+mn^{2/3})$, where the $\tilde{O}$ notation suppresses polylogarithmic factors in $n$, the desired accuracy, and the appropriate condition number (i.e. the mixing time or restart probability). Our result improves upon the previous fastest running times for these problems; previous results either invoke a general purpose linear system solver on a $n\times n$ matrix with $m$ non-zero entries, or depend polynomially on the desired error or natural condition number associated with the problem (i.e. the mixing time or restart probability). For sparse graphs, we obtain a running time of $\tilde{O}(n^{7/4})$, breaking the $O(n^{2})$ barrier of the best running time one could hope to achieve using fast matrix multiplication. We achieve our result by providing a similar running time improvement for solving directed Laplacian systems, a natural directed or asymmetric analog of the well studied symmetric or undirected Laplacian systems. We show how to solve such systems in time $\tilde{O}(m^{3/4}n+mn^{2/3})$, and efficiently reduce a broad range of problems to solving $\tilde{O}(1)$ directed Laplacian systems on Eulerian graphs. We hope these results and our analysis open the door for further study into directed spectral graph theory.

Accelerated spectral clustering using graph filtering of random signals

Abstract: We build upon recent advances in graph signal processing to propose a faster spectral clustering algorithm. Indeed, classical spectral clustering is based on the computation of the first k eigenvectors of the similarity matrix' Laplacian, whose computation cost, even for sparse matrices, becomes prohibitive for large datasets. We show that we can estimate the spectral clustering distance matrix without computing these eigenvectors: by graph filtering random signals. Also, we take advantage of the stochasticity of these random vectors to estimate the number of clusters k. We compare our method to classical spectral clustering on synthetic data, and show that it reaches equal performance while being faster by a factor at least two for large datasets.

Structural and Spectral Properties of Deterministic Aperiodic Optical Structures

Abstract: In this comprehensive paper we have addressed structure-property relationships in a number of representative systems with periodic, random, quasi-periodic and deterministic aperiodic geometry using the interdisciplinary methods of spatial point pattern analysis and spectral graph theory as well as the rigorous Green’s matrix method, which provides access to the electromagnetic scattering behavior and spectral fluctuations (distributions of complex eigenvalues as well as of their level spacing) of deterministic aperiodic optical media for the first time.

Spectral graph sparsification in nearly-linear time leveraging efficient spectral perturbation analysis

Abstract: Spectral graph sparsification aims to find an ultra-sparse subgraph whose Laplacian matrix can well approximate the original Laplacian matrix in terms of its eigenvalues and eigenvectors. The resultant sparsified subgraph can be efficiently leveraged as a proxy in a variety of numerical computation applications and graph-based algorithms. This paper introduces a practically efficient, nearly-linear time spectral graph sparsification algorithm that can immediately lead to the development of nearly-linear time symmetric diagonally-dominant (SDD) matrix solvers. Our spectral graph sparsi-fication algorithm can efficiently build an ultra-sparse subgraph from a spanning tree subgraph by adding a few “spectrally-critical” off-tree edges back to the spanning tree, which is enabled by a novel spectral perturbation approach and allows to approximately preserve key spectral properties of the original graph Laplacian. Extensive experimental results confirm the nearly-linear runtime scalability of an SDD matrix solver for large-scale, real-world problems, such as VLSI, thermal and finite-element analysis problems, etc. For instance, a sparse SDD matrix with 40 million unknowns and 180 million nonzeros can be solved (1E-3 accuracy level) within two minutes using a single CPU core and about 6GB memory.

Quantum graphs which optimize the spectral gap

Abstract: A finite discrete graph is turned into a quantum (metric) graph once a finite length is assigned to each edge and the one-dimensional Laplacian is taken to be the operator. We study the dependence of the spectral gap (the first positive Laplacian eigenvalue) on the choice of edge lengths. In particular, starting from a certain discrete graph, we seek the quantum graph for which an optimal (either maximal or minimal) spectral gap is obtained. We fully solve the minimization problem for all graphs. We develop tools for investigating the maximization problem and solve it for some families of graphs.

Nonlinear Laplacian for Digraphs and its Applications to Network Analysis

Abstract: In this work, we introduce a new Markov operator associated with a digraph, which we refer to as a nonlinear Laplacian. Unlike previous Laplacians for digraphs, the nonlinear Laplacian does not rely on the stationary distribution of the random walk process and is well defined on digraphs that are not strongly connected. We show that the nonlinear Laplacian has nontrivial eigenvalues and give a Cheeger-like inequality, which relates the conductance of a digraph and the smallest non-zero eigenvalue of its nonlinear Laplacian. Finally, we apply the nonlinear Laplacian to the analysis of real-world networks and obtain encouraging results.

Network topology identification from spectral templates

Abstract: Network topology inference is a cornerstone problem in statistical analyses of complex systems. In this context, the fresh look advocated here permeates benefits from convex optimization and graph signal processing, to identify the so-termed graph shift operator (encoding the network topology) given only the eigenvectors of the shift. These spectral templates can be obtained, for example, from principal component analysis of a set of graph signals defined on the particular network. The novel idea is to find a graph shift that while being consistent with the provided spectral information, it endows the network structure with certain desired properties such as sparsity. The focus is on developing efficient recovery algorithms along with identifiability conditions for two particular shifts, the adjacency matrix and the normalized graph Laplacian. Application domains include network topology identification from steady-state signals generated by a diffusion process, and design of a graph filter that facilitates the distributed implementation of a prescribed linear network operator. Numerical tests showcase the effectiveness of the proposed algorithms in recovering synthetic and structural brain networks.

Connectivity, toughness, spanning trees of bounded degree, and the spectrum of regular graphs

Abstract: The eigenvalues of graphs are related to many of its combinatorial properties. In his fundamental work, Fiedler showed the close connections between the Laplacian eigenvalues and eigenvectors of a graph and its vertex-connectivity and edge-connectivity. We present some new results describing the connections between the spectrum of a regular graph and other combinatorial parameters such as its generalized connectivity, toughness, and the existence of spanning trees with bounded degree.

A review of geometric, topological and graph theory apparatuses for the modeling and analysis of biomolecular data

Abstract: Geometric, topological and graph theory modeling and analysis of biomolecules are of essential importance in the conceptualization of molecular structure, function, dynamics, and transport. On the one hand, geometric modeling provides molecular surface and structural representation, and offers the basis for molecular visualization, which is crucial for the understanding of molecular structure and interactions. On the other hand, it bridges the gap between molecular structural data and theoretical/mathematical models. Topological analysis and modeling give rise to atomic critical points and connectivity, and shed light on the intrinsic topological invariants such as independent components (atoms), rings (pockets) and cavities. Graph theory analyzes biomolecular interactions and reveals biomolecular structure-function relationship. In this paper, we review certain geometric, topological and graph theory apparatuses for biomolecular data modeling and analysis. These apparatuses are categorized into discrete and continuous ones. For discrete approaches, graph theory, Gaussian network model, anisotropic network model, normal mode analysis, quasi-harmonic analysis, flexibility and rigidity index, molecular nonlinear dynamics, spectral graph theory, and persistent homology are discussed. For continuous mathematical tools, we present discrete to continuum mapping, high dimensional persistent homology, biomolecular geometric modeling, differential geometry theory of surfaces, curvature evaluation, variational derivation of minimal molecular surfaces, atoms in molecule theory and quantum chemical topology. Four new approaches, including analytical minimal molecular surface, Hessian matrix eigenvalue map, curvature map and virtual particle model, are introduced for the first time to bridge the gaps in biomolecular modeling and analysis.

Subsampling for Graph Power Spectrum Estimation

Abstract: In this paper we focus on subsampling stationary random processes that reside on the vertices of undirected graphs. Second-order stationary graph signals are obtained by filtering white noise and they admit a well-defined power spectrum. Estimating the graph power spectrum forms a central component of stationary graph signal processing and related inference tasks. We show that by sampling a significantly smaller subset of vertices and using simple least squares, we can reconstruct the power spectrum of the graph signal from the subsampled observations, without any spectral priors. In addition, a near-optimal greedy algorithm is developed to design the subsampling scheme.