Showing papers on "Spectral graph theory published in 2017"

Graph Learning From Data Under Laplacian and Structural Constraints

Abstract: Graphs are fundamental mathematical structures used in various fields to represent data, signals, and processes In this paper, we propose a novel framework for learning/estimating graphs from data The proposed framework includes (i) formulation of various graph learning problems, (ii) their probabilistic interpretations, and (iii) associated algorithms Specifically, graph learning problems are posed as the estimation of graph Laplacian matrices from some observed data under given structural constraints (eg, graph connectivity and sparsity level) From a probabilistic perspective, the problems of interest correspond to maximum a posteriori parameter estimation of Gaussian–Markov random field models, whose precision (inverse covariance) is a graph Laplacian matrix For the proposed graph learning problems, specialized algorithms are developed by incorporating the graph Laplacian and structural constraints The experimental results demonstrate that the proposed algorithms outperform the current state-of-the-art methods in terms of accuracy and computational efficiency

Stationary Signal Processing on Graphs

Abstract: Graphs are a central tool in machine learning and information processing as they allow to conveniently capture the structure of complex datasets. In this context, it is of high importance to develop flexible models of signals defined over graphs or networks. In this paper, we generalize the traditional concept of wide sense stationarity to signals defined over the vertices of arbitrary weighted undirected graphs. We show that stationarity is expressed through the graph localization operator reminiscent of translation. We prove that stationary graph signals are characterized by a well-defined power spectral density that can be efficiently estimated even for large graphs. We leverage this new concept to derive Wiener-type estimation procedures of noisy and partially observed signals and illustrate the performance of this new model for denoising and regression.

Graph Laplacian Regularization for Image Denoising: Analysis in the Continuous Domain

Abstract: Inverse imaging problems are inherently underdetermined, and hence, it is important to employ appropriate image priors for regularization. One recent popular prior—the graph Laplacian regularizer—assumes that the target pixel patch is smooth with respect to an appropriately chosen graph. However, the mechanisms and implications of imposing the graph Laplacian regularizer on the original inverse problem are not well understood. To address this problem, in this paper, we interpret neighborhood graphs of pixel patches as discrete counterparts of Riemannian manifolds and perform analysis in the continuous domain, providing insights into several fundamental aspects of graph Laplacian regularization for image denoising. Specifically, we first show the convergence of the graph Laplacian regularizer to a continuous-domain functional, integrating a norm measured in a locally adaptive metric space. Focusing on image denoising, we derive an optimal metric space assuming non-local self-similarity of pixel patches, leading to an optimal graph Laplacian regularizer for denoising in the discrete domain. We then interpret graph Laplacian regularization as an anisotropic diffusion scheme to explain its behavior during iterations, e.g., its tendency to promote piecewise smooth signals under certain settings. To verify our analysis, an iterative image denoising algorithm is developed. Experimental results show that our algorithm performs competitively with state-of-the-art denoising methods, such as BM3D for natural images, and outperforms them significantly for piecewise smooth images.

Distance Metric Learning using Graph Convolutional Networks: Application to Functional Brain Networks

Abstract: Evaluating similarity between graphs is of major importance in several computer vision and pattern recognition problems, where graph representations are often used to model objects or interactions between elements. The choice of a distance or similarity metric is, however, not trivial and can be highly dependent on the application at hand. In this work, we propose a novel metric learning method to evaluate distance between graphs that leverages the power of convolutional neural networks, while exploiting concepts from spectral graph theory to allow these operations on irregular graphs. We demonstrate the potential of our method in the field of connectomics, where neuronal pathways or functional connections between brain regions are commonly modelled as graphs. In this problem, the definition of an appropriate graph similarity function is critical to unveil patterns of disruptions associated with certain brain disorders. Experimental results on the ABIDE dataset show that our method can learn a graph similarity metric tailored for a clinical application, improving the performance of a simple k-nn classifier by 11.9% compared to a traditional distance metric.

On the Graph Fourier Transform for Directed Graphs

Abstract: The analysis of signals defined over a graph is relevant in many applications, such as social and economic networks, big data or biological networks, and so on. A key tool for analyzing these signals is the so-called graph Fourier transform (GFT). Alternative definitions of GFT have been suggested in the literature, based on the eigen-decomposition of either the graph Laplacian or adjacency matrix. In this paper, we address the general case of directed graphs and we propose an alternative approach that builds the graph Fourier basis as the set of orthonormal vectors that minimize a continuous extension of the graph cut size, known as the Lovasz extension. To cope with the nonconvexity of the problem, we propose two alternative iterative optimization methods, properly devised for handling orthogonality constraints. Finally, we extend the method to minimize a continuous relaxation of the balanced cut size. The formulated problem is again nonconvex, and we propose an efficient solution method based on an explicit–implicit gradient algorithm.

Distance metric learning using graph convolutional networks: application to functional brain networks

Abstract: Evaluating similarity between graphs is of major importance in several computer vision and pattern recognition problems, where graph representations are often used to model objects or interactions between elements. The choice of a distance or similarity metric is, however, not trivial and can be highly dependent on the application at hand. In this work, we propose a novel metric learning method to evaluate distance between graphs that leverages the power of convolutional neural networks, while exploiting concepts from spectral graph theory to allow these operations on irregular graphs. We demonstrate the potential of our method in the field of connectomics, where neuronal pathways or functional connections between brain regions are commonly modelled as graphs. In this problem, the definition of an appropriate graph similarity function is critical to unveil patterns of disruptions associated with certain brain disorders. Experimental results on the ABIDE dataset show that our method can learn a graph similarity metric tailored for a clinical application, improving the performance of a simple k-nn classifier by 11.9% compared to a traditional distance metric.

Coherent structure colouring: identification of coherent structures from sparse data using graph theory

Abstract: We present a frame-invariant method for detecting coherent structures from Lagrangian flow trajectories that can be sparse in number, as is the case in many fluid mechanics applications of practical interest. The method, based on principles used in graph colouring and spectral graph drawing algorithms, examines a measure of the kinematic dissimilarity of all pairs of fluid trajectories, measured either experimentally, e.g. using particle tracking velocimetry, or numerically, by advecting fluid particles in the Eulerian velocity field. Coherence is assigned to groups of particles whose kinematics remain similar throughout the time interval for which trajectory data are available, regardless of their physical proximity to one another. Through the use of several analytical and experimental validation cases, this algorithm is shown to robustly detect coherent structures using significantly less flow data than are required by existing spectral graph theory methods.

Almost-linear-time algorithms for Markov chains and new spectral primitives for directed graphs

Abstract: In this paper, we begin to address the longstanding algorithmic gap between general and reversible Markov chains. We develop directed analogues of several spectral graph-theoretic tools that had previously been available only in the undirected setting, and for which it was not clear that directed versions even existed. In particular, we provide a notion of approximation for directed graphs, prove sparsifiers under this notion always exist, and show how to construct them in almost linear time. Using this notion of approximation, we design the first almost-linear-time directed Laplacian system solver, and, by leveraging the recent framework of [Cohen-Kelner-Peebles-Peng-Sidford-Vladu, FOCS '16], we also obtain almost-linear-time algorithms for computing the stationary distribution of a Markov chain, computing expected commute times in a directed graph, and more. For each problem, our algorithms improve the previous best running times of O((nm3/4 + n2/3 m) logO(1) (n κ e-1)) to O((m + n2O(√lognloglogn)) logO(1) (n κe-1)) where n is the number of vertices in the graph, m is the number of edges, κ is a natural condition number associated with the problem, and e is the desired accuracy. We hope these results open the door for further studies into directed spectral graph theory, and that they will serve as a stepping stone for designing a new generation of fast algorithms for directed graphs.

An SDP-based algorithm for linear-sized spectral sparsification

Abstract: For any undirected and weighted graph G=(V,E,w) with n vertices and m edges, we call a sparse subgraph H of G, with proper reweighting of the edges, a (1+Iµ)-spectral sparsifier if (1-e)xTLGx≤xT LH x≤(1+e)xTLGx holds for any xEℝn, where LG and LH are the respective Laplacian matrices of G and H. Noticing that Ω(m) time is needed for any algorithm to construct a spectral sparsifier and a spectral sparsifier of G requires Ω(n) edges, a natural question is to investigate, for any constant e, if a (1+e)-spectral sparsifier of G with O(n) edges can be constructed in Ο(m) time, where the Ο notation suppresses polylogarithmic factors. All previous constructions on spectral sparsification require either super-linear number of edges or m1+Ω(1) time. In this work we answer this question affirmatively by presenting an algorithm that, for any undirected graph G and e>0, outputs a (1+e)-spectral sparsifier of G with O(n/e2) edges in Ο(m/eO(1)) time. Our algorithm is based on three novel techniques: (1) a new potential function which is much easier to compute yet has similar guarantees as the potential functions used in previous references; (2) an efficient reduction from a two-sided spectral sparsifier to a one-sided spectral sparsifier; (3) constructing a one-sided spectral sparsifier by a semi-definite program.

Network topology inference from non-stationary graph signals

Abstract: We address the problem of inferring a graph from nodal observations, which are modeled as non-stationary graph signals generated by local diffusion dynamics that depend on the structure of the sought network. Using the so-called graph-shift operator (GSO) as a matrix representation of the graph, we first identify the eigenvectors of the shift matrix from realizations of the diffused signals, and then we rely on these spectral templates to estimate the eigenvalues by imposing desirable properties on the graph to be recovered. Different from the stationary setting where the GSO and the covariance matrix of the observed signals are simultaneously diagonalizable, here they are not. Hence, estimating the eigenvectors requires first estimating the unknown diffusion (graph) filter - a polynomial in the GSO which does preserve the sought eigenbasis. To carry out this initial system identification step, we leverage different sources of information on the input signal driving the diffusion process on the graph. Numerical tests showcase the effectiveness of the proposed algorithms in recovering social and structural brain graphs.

Rotation Averaging and Strong Duality

Abstract: In this paper we explore the role of duality principles within the problem of rotation averaging, a fundamental task in a wide range of computer vision applications. In its conventional form, rotation averaging is stated as a minimization over multiple rotation constraints. As these constraints are non-convex, this problem is generally considered challenging to solve globally. We show how to circumvent this difficulty through the use of Lagrangian duality. While such an approach is well-known it is normally not guaranteed to provide a tight relaxation. Based on spectral graph theory, we analytically prove that in many cases there is no duality gap unless the noise levels are severe. This allows us to obtain certifiably global solutions to a class of important non-convex problems in polynomial time. We also propose an efficient, scalable algorithm that out-performs general purpose numerical solvers and is able to handle the large problem instances commonly occurring in structure from motion settings. The potential of this proposed method is demonstrated on a number of different problems, consisting of both synthetic and real-world data.

Edge connectivity and the spectral gap of combinatorial and quantum graphs

Abstract: We derive a number of upper and lower bounds for the first nontrivial eigenvalue of Laplacians on combinatorial and quantum graph in terms of the edge connectivity, i.e. the minimal number of edges which need to be removed to make the graph disconnected. On combinatorial graphs, one of the bounds corresponds to a well-known inequality of Fiedler, of which we give a new variational proof. On quantum graphs, the corresponding bound generalizes a recent result of Band and Levy. All proofs are general enough to yield corresponding estimates for the p-Laplacian and allow us to identify the minimizers. Based on the Betti number of the graph, we also derive upper and lower bounds on all eigenvalues which are 'asymptotically correct', i.e. agree with the Weyl asymptotics for the eigenvalues of the quantum graph. In particular, the lower bounds improve the bounds of Friedlander on any given graph for all but finitely many eigenvalues, while the upper bounds improve recent results of Ariturk. Our estimates are also used to derive bounds on the eigenvalues of the normalized Laplacian matrix that improve known bounds of spectral graph theory.

Network-theoretic approach to sparsified discrete vortex dynamics

Abstract: We examine discrete vortex dynamics in two-dimensional flow through a network-theoretic approach. The interaction of the vortices is represented with a graph, which allows the use of network-theoretic approaches to identify key vortex-to-vortex interactions. We employ sparsification techniques on these graph representations based on spectral theory for constructing sparsified models and evaluating the dynamics of vortices in the sparsified setup. Identification of vortex structures based on graph sparsification and sparse vortex dynamics are illustrated through an example of point-vortex clusters interacting amongst themselves. We also evaluate the performance of sparsification with increasing number of point vortices. The sparsified-dynamics model developed with spectral graph theory requires reduced number of vortex-to-vortex interactions but agrees well with the full nonlinear dynamics. Furthermore, the sparsified model derived from the sparse graphs conserves the invariants of discrete vortex dynamics. We highlight the similarities and differences between the present sparsified-dynamics model and the reduced-order models.

New concepts of vague graphs

Abstract: The concept of vague graph was introduced by Ramakrishna (Int J Comput Cognit 7:51–58, 2009). Since the vague models give more precision, flexibility, and compatibility to the system as compared to the classical and fuzzy models, in this paper, the concept of energy of fuzzy graph is extended to the energy of a vague graph. It has many applications in physics, chemistry, computer science, and other branches of mathematics. We define adjacency matrix, degree matrix, laplacian matrix, spectrum, and energy of a vague graph in terms of their adjacency matrix. The spectrum of a vague graph appears in physics statistical problems, and combinatorial optimization problems in mathematics. Also, the lower and upper bounds for the energy of a vague graph are also derived. Finally, we give some applications of energy in vague graph and other sciences.

Inferring sparse graphs from smooth signals with theoretical guarantees

Abstract: We consider the problem of inferring a graph from signals which are assumed to be smooth over the graph, in the setting where the graph is also assumed to be sparse. We focus on the case where measurements are Gaussian vectors and the graph topology is encoded in the inverse of the covariance matrix. In addition, the weights of the inverse covariance are assumed to be such that the model is attractive—all partial correlations are non-negative. Unlike other approaches which seek to minimize the Laplacian quadratic form or involve solving a log-det program, we study a simple estimator based on soft thresholding. The estimator involves computing only a single eigenvalue decomposition, and so it can easily scale to networks with thousands of vertices. We provide theoretical results on the reconstruction error as a function of the number of observations and problem dimensions for the case where the underlying graph is assumed to be sparse.

The mutually beneficial relationship of graphs and matrices

Abstract: Graphs and matrices enjoy a fascinating and mutually beneficial relationship. This interplay has benefited both graph theory and linear algebra. In one direction, knowledge about one of the graphs that can be associated with a matrix can be used to illuminate matrix properties and to get better information about the matrix. Examples include the use of digraphs to obtain strong results on diagonal dominance and eigenvalue inclusion regions and the use of the Rado-Hall theorem to deduce properties of special classes of matrices. Going the other way, linear algebraic properties of one of the matrices associated with a graph can be used to obtain useful combinatorial information about the graph. The adjacency matrix and the Laplacian matrix are two well-known matrices associated to a graph, and their eigenvalues encode important information about the graph. Another important linear algebraic invariant associated with a graph is the Colin de Verdiere number, which, for instance, characterises certain topological properties of the graph. This book is not a comprehensive study of graphs and matrices. The particular content of the lectures was chosen for its accessibility, beauty, and current relevance, and for the possibility of enticing the audience to want to learn more.

Network inference from consensus dynamics

Abstract: We consider the problem of identifying the topology of a weighted, undirected network G from observing snapshots of multiple independent consensus dynamics. Specifically, we observe the opinion profiles of a group of agents for a set of M independent topics and our goal is to recover the precise relationships between the agents, as specified by the unknown network G. In order to overcome the under-determinacy of the problem at hand, we leverage concepts from spectral graph theory and convex optimization to unveil the underlying network structure. More precisely, we formulate the network inference problem as a convex optimization that seeks to endow the network with certain desired properties — such as sparsity — while being consistent with the spectral information extracted from the observed opinions. This is complemented with theoretical results proving consistency as the number M of topics grows large. We further illustrate our method by numerical experiments, which showcase the effectiveness of the technique in recovering synthetic and real-world networks.

Edge connectivity and the spectral gap of combinatorial and quantum graphs

Abstract: We derive a number of upper and lower bounds for the first nontrivial eigenvalue of a finite quantum graph in terms of the edge connectivity of the graph, i.e., the minimal number of edges which need to be removed to make the graph disconnected. On combinatorial graphs, one of the bounds is the well-known inequality of Fiedler, of which we give a new variational proof. On quantum graphs, the corresponding bound generalizes a recent result of Band and L\'evy. All proofs are general enough to yield corresponding estimates for the $p$-Laplacian and allow us to identify the minimizers. Based on the Betti number of the graph, we also derive upper and lower bounds on all eigenvalues which are "asymptotically correct", i.e. agree with the Weyl asymptotics for the eigenvalues of the quantum graph. In particular, the lower bounds improve the bounds of Friedlander on any given graph for all but finitely many eigenvalues, while the upper bounds improve recent results of Ariturk. Our estimates are also used to derive bounds on the eigenvalues of the normalized Laplacian matrix that improve known bounds of spectral graph theory.

The Complexity of Translationally Invariant Spin Chains with Low Local Dimension

Abstract: We prove that estimating the ground state energy of a translationally invariant, nearest-neighbour Hamiltonian on a 1D spin chain is $$\textsf {QMA}_{{\textsf {EXP}}}$$ -complete, even for systems of low local dimension ( $$\approx 40$$ ). This is an improvement over the best previously known result by several orders of magnitude, and it shows that spin-glass-like frustration can occur in translationally invariant quantum systems with a local dimension comparable to the smallest-known non-translationally invariant systems with similar behaviour. While previous constructions of such systems rely on standard models of quantum computation, we construct a new model that is particularly well-suited for encoding quantum computation into the ground state of a translationally invariant system. This allows us to shift the proof burden from optimizing the Hamiltonian encoding a standard computational model, to proving universality of a simple model. Previous techniques for encoding quantum computation into the ground state of a local Hamiltonian allow only a linear sequence of gates, hence only a linear (or nearly linear) path in the graph of all computational states. We extend these techniques by allowing significantly more general paths, including branching and cycles, thus enabling a highly efficient encoding of our computational model. However, this requires more sophisticated techniques for analysing the spectrum of the resulting Hamiltonian. To address this, we introduce a framework of graphs with unitary edge labels. After relating our Hamiltonian to the Laplacian of such a unitary labelled graph, we analyse its spectrum by combining matrix analysis and spectral graph theory techniques.

Cheeger Inequalities for Submodular Transformations.

Abstract: The Cheeger inequality for undirected graphs, which relates the conductance of an undirected graph and the second smallest eigenvalue of its normalized Laplacian, is a cornerstone of spectral graph theory. The Cheeger inequality has been extended to directed graphs and hypergraphs using normalized Laplacians for those, that are no longer linear but piecewise linear transformations. In this paper, we introduce the notion of a submodular transformation $F:\{0,1\}^n \to \mathbb{R}^m$, which applies $m$ submodular functions to the $n$-dimensional input vector, and then introduce the notions of its Laplacian and normalized Laplacian. With these notions, we unify and generalize the existing Cheeger inequalities by showing a Cheeger inequality for submodular transformations, which relates the conductance of a submodular transformation and the smallest non-trivial eigenvalue of its normalized Laplacian. This result recovers the Cheeger inequalities for undirected graphs, directed graphs, and hypergraphs, and derives novel Cheeger inequalities for mutual information and directed information. Computing the smallest non-trivial eigenvalue of a normalized Laplacian of a submodular transformation is NP-hard under the small set expansion hypothesis. In this paper, we present a polynomial-time $O(\log n)$-approximation algorithm for the symmetric case, which is tight, and a polynomial-time $O(\log^2n+\log n \cdot \log m)$-approximation algorithm for the general case. We expect the algebra concerned with submodular transformations, or \emph{submodular algebra}, to be useful in the future not only for generalizing spectral graph theory but also for analyzing other problems that involve piecewise linear transformations, e.g., deep learning.

Efficient quantum algorithms for analyzing large sparse electrical networks

Abstract: Analyzing large sparse electrical networks is a fundamental task in physics, electrical engineering and computer science. We propose two classes of quantum algorithms for this task. The first class is based on solving linear systems, and the second class is based on using quantum walks. These algorithms compute various electrical quantities, including voltages, currents, dissipated powers and effective resistances, in time poly(d,c,log(N),1/λ,1/e), where N is the number of vertices in the network, d is the maximum unweighted degree of the vertices, c is the ratio of largest to smallest edge resistance, λ is the spectral gap of the normalized Laplacian of the network, and e is the accuracy. Furthermore, we show that the polynomial dependence on 1/λ is necessary. This implies that our algorithms are optimal up to polynomial factors and cannot be significantly improved.

Monotonicity properties and spectral characterization of power redistribution in cascading failures

Abstract: In this work, we apply spectral graph theory methods to study the monotonicity and structural properties of power redistribution in a cascading failure process. We demonstrate that in contrast to the lack of monotonicity in physical domain, there is a rich collection of monotonicity one can explore in the spectral domain, leading to a systematic way to define topological metrics that are monotonic. It is further shown that many useful quantities in cascading failure analysis can be unified into a spectral inner product, which itself is related to graphical properties of the transmission network. Such graphical interpretations precisely capture the Kirchhoff's law expressed in terms of graph structural properties and gauge the impact of a line when it is tripped. We illustrate that our characterization leads to a tree-partition of the network so that failure cascading can be localized.

Spectral identification of networks using sparse measurements

Abstract: We propose a new method to recover global information about a network of interconnected dynamical systems based on observations made at a small number (possibly one) of its nodes. In contrast to classical identification of full graph topology, we focus on the identification of the spectral graph-theoretic properties of the network, a framework that we call spectral network identification. The main theoretical results connect the spectral properties of the network to the spectral properties of the dynamics, which are well-defined in the context of the so-called Koopman operator and can be extracted from data through the dynamic mode decomposition algorithm. These results are obtained for networks of diffusively-coupled units that admit a stable equilibrium state. For large networks, a statistical approach is considered, which focuses on spectral moments of the network and is well-suited to the case of heterogeneous populations. Our framework provides efficient numerical methods to infer global information ...