Showing papers on "Spectral graph theory published in 2020"

Graph Representation Learning

Abstract: Graph-structured data is ubiquitous throughout the natural and social sciences, from telecommunication networks to quantum chemistry. Building relational inductive biases into deep learnin...

Articulated Shape Matching Using Laplacian Eigenfunctions and Unsupervised Point Registration

Abstract: Matching articulated shapes represented by voxel-sets reduces to maximal sub-graph isomorphism when each set is described by a weighted graph. Spectral graph theory can be used to map these graphs onto lower dimensional spaces and match shapes by aligning their embeddings in virtue of their invariance to change of pose. Classical graph isomorphism schemes relying on the ordering of the eigenvalues to align the eigenspaces fail when handling large data-sets or noisy data. We derive a new formulation that finds the best alignment between two congruent $K$-dimensional sets of points by selecting the best subset of eigenfunctions of the Laplacian matrix. The selection is done by matching eigenfunction signatures built with histograms, and the retained set provides a smart initialization for the alignment problem with a considerable impact on the overall performance. Dense shape matching casted into graph matching reduces then, to point registration of embeddings under orthogonal transformations; the registration is solved using the framework of unsupervised clustering and the EM algorithm. Maximal subset matching of non identical shapes is handled by defining an appropriate outlier class. Experimental results on challenging examples show how the algorithm naturally treats changes of topology, shape variations and different sampling densities.

GraphZoom: A Multi-level Spectral Approach for Accurate and Scalable Graph Embedding

Abstract: Graph embedding techniques have been increasingly deployed in a multitude of different applications that involve learning on non-Euclidean data. However, existing graph embedding models either fail to incorporate node attribute information during training or suffer from node attribute noise, which compromises the accuracy. Moreover, very few of them scale to large graphs due to their high computational complexity and memory usage. In this paper we propose GraphZoom, a multi-level framework for improving both accuracy and scalability of unsupervised graph embedding algorithms. GraphZoom first performs graph fusion to generate a new graph that effectively encodes the topology of the original graph and the node attribute information. This fused graph is then repeatedly coarsened into a much smaller graph by merging nodes with high spectral similarities. GraphZoom allows any existing embedding methods to be applied to the coarsened graph, before it progressively refine the embeddings obtained at the coarsest level to increasingly finer graphs. We have evaluated our approach on a number of popular graph datasets for both transductive and inductive tasks. Our experiments show that GraphZoom increases the classification accuracy and significantly reduces the run time compared to state-of-the-art unsupervised embedding methods.

Spectral graph theory of brain oscillations

Abstract: The relationship between the brain's structural wiring and the functional patterns of neural activity is of fundamental interest in computational neuroscience. We examine a hierarchical, linear graph spectral model of brain activity at mesoscopic and macroscopic scales. The model formulation yields an elegant closed-form solution for the structure-function problem, specified by the graph spectrum of the structural connectome's Laplacian, with simple, universal rules of dynamics specified by a minimal set of global parameters. The resulting parsimonious and analytical solution stands in contrast to complex numerical simulations of high dimensional coupled nonlinear neural field models. This spectral graph model accurately predicts spatial and spectral features of neural oscillatory activity across the brain and was successful in simultaneously reproducing empirically observed spatial and spectral patterns of alpha-band (8-12 Hz) and beta-band (15-30 Hz) activity estimated from source localized magnetoencephalography (MEG). This spectral graph model demonstrates that certain brain oscillations are emergent properties of the graph structure of the structural connectome and provides important insights towards understanding the fundamental relationship between network topology and macroscopic whole-brain dynamics. .

A Unified Framework for Structured Graph Learning via Spectral Constraints

Abstract: Graph learning from data represents a canonical problem that has received substantial attention in the literature. However, insufficient work has been done in incorporating prior structural knowledge onto the learning of underlying graphical models from data. Learning a graph with a specific structure is essential for interpretability and identification of the relationships among data. Useful structured graphs include the multi-component graph, bipartite graph, connected graph, sparse graph, and regular graph. In general, structured graph learning is an NP-hard combinatorial problem, therefore, designing a general tractable optimization method is extremely challenging. In this paper, we introduce a unified graph learning framework lying at the integration of Gaussian graphical models and spectral graph theory. To impose a particular structure on a graph, we first show how to formulate the combinatorial constraints as an analytical property of the graph matrix. Then we develop an optimization framework that leverages graph learning with specific structures via spectral constraints on graph matrices. The proposed algorithms are provably convergent, computationally efficient, and practically amenable for numerous graph-based tasks. Extensive numerical experiments with both synthetic and real data sets illustrate the effectiveness of the proposed algorithms. The code for all the simulations is made available as an open source repository.

Persistent Laplacians: properties, algorithms and implications.

Abstract: We present a thorough study of the theoretical properties and devise efficient algorithms for the \emph{persistent Laplacian}, an extension of the standard combinatorial Laplacian to the setting of pairs (or, in more generality, sequences) of simplicial complexes $K \hookrightarrow L$, which was recently introduced by Wang, Nguyen, and Wei. In particular, in analogy with the non-persistent case, we first prove that the nullity of the $q$-th persistent Laplacian $\Delta_q^{K,L}$ equals the $q$-th persistent Betti number of the inclusion $(K \hookrightarrow L)$. We then present an initial algorithm for finding a matrix representation of $\Delta_q^{K,L}$, which itself helps interpret the persistent Laplacian. We exhibit a novel relationship between the persistent Laplacian and the notion of Schur complement of a matrix which has several important implications. In the graph case, it both uncovers a link with the notion of effective resistance and leads to a persistent version of the Cheeger inequality. This relationship also yields an additional, very simple algorithm for finding (a matrix representation of) the $q$-th persistent Laplacian which in turn leads to a novel and fundamentally different algorithm for computing the $q$-th persistent Betti number for a pair $(K,L)$ which can be significantly more efficient than standard algorithms. Finally, we study persistent Laplacians for simplicial filtrations and present novel stability results for their eigenvalues. Our work brings methods from spectral graph theory, circuit theory, and persistent homology together with a topological view of the combinatorial Laplacian on simplicial complexes.

Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks.

Abstract: Deep learning's success has been widely recognized in a variety of machine learning tasks, including image classification, audio recognition, and natural language processing. As an extension of deep learning beyond these domains, graph neural networks (GNNs) are designed to handle the non-Euclidean graph-structure which is intractable to previous deep learning techniques. Existing GNNs are presented using various techniques, making direct comparison and cross-reference more complex. Although existing studies categorize GNNs into spatial-based and spectral-based techniques, there hasn't been a thorough examination of their relationship. To close this gap, this study presents a single framework that systematically incorporates most GNNs. We organize existing GNNs into spatial and spectral domains, as well as expose the connections within each domain. A review of spectral graph theory and approximation theory builds a strong relationship across the spatial and spectral domains in further investigation.

Exploring Structure-Adaptive Graph Learning for Robust Semi-Supervised Classification

Abstract: Graph Convolutional Neural Networks (GCNNs) are generalizations of CNNs to graph-structured data, in which convolution is guided by the graph topology. In many cases where graphs are unavailable, existing methods manually construct graphs or learn task-driven adaptive graphs. In this paper, we propose Graph Learning Neural Networks (GLNNs), which exploit the optimization of graphs (the adjacency matrix in particular) from both data and tasks. Leveraging on spectral graph theory, we propose the objective of graph learning from a sparsity constraint, properties of a valid adjacency matrix as well as a graph Laplacian regularizer via maximum a posteriori estimation. The optimization objective is then integrated into the loss function of the GCNN, which adapts the graph topology to not only labels of a specific task but also the input data. Experimental results show that our proposed GLNN significantly outperforms state-of-the-art approaches over widely adopted social network datasets and citation network datasets for semi-supervised classification.

A nonparametric approach to high-dimensional k-sample comparison problems

Abstract: SummaryHigh-dimensional $k$-sample comparison is a common task in applications. We construct a class of easy-to-implement distribution-free tests based on new nonparametric tools and unexplored connections with spectral graph theory. The test is shown to have various desirable properties and a characteristic exploratory flavour that has practical consequences for statistical modelling. Numerical examples show that the proposed method works surprisingly well across a broad range of realistic situations.

Synchrony and Complexity in State-Related EEG Networks: An Application of Spectral Graph Theory.

Abstract: The brain may be considered as a synchronized dynamic network with several coherent dynamical units. However, concerns remain whether synchronizability is a stable state in the brain networks. If so, which index can best reveal the synchronizability in brain networks? To answer these questions, we tested the application of the spectral graph theory and the Shannon entropy as alternative approaches in neuroimaging. We specifically tested the alpha rhythm in the resting-state eye closed (rsEC) and the resting-state eye open (rsEO) conditions, a well-studied classical example of synchrony in neuroimaging EEG. Since the synchronizability of alpha rhythm is more stable during the rsEC than the rsEO, we hypothesized that our suggested spectral graph theory indices (as reliable measures to interpret the synchronizability of brain signals) should exhibit higher values in the rsEC than the rsEO condition. We performed two separate analyses of two different datasets (as elementary and confirmatory studies). Based on the results of both studies and in agreement with our hypothesis, the spectral graph indices revealed higher stability of synchronizability in the rsEC condition. The k-mean analysis indicated that the spectral graph indices can distinguish the rsEC and rsEO conditions by considering the synchronizability of brain networks. We also computed correlations among the spectral indices, the Shannon entropy, and the topological indices of brain networks, as well as random networks. Correlation analysis indicated that although the spectral and the topological properties of random networks are completely independent, these features are significantly correlated with each other in brain networks. Furthermore, we found that complexity in the investigated brain networks is inversely related to the stability of synchronizability. In conclusion, we revealed that the spectral graph theory approach can be reliably applied to study the stability of synchronizability of state-related brain networks.

Graph Coarsening with Preserved Spectral Properties

Abstract: Large-scale graphs are widely used to represent object relationships in many real world applications The occurrence of large-scale graphs presents significant computational challenges to process, analyze, and extract information Graph coarsening techniques are commonly used to reduce the computational load while attempting to maintain the basic structural properties of the original graph As there is no consensus on the specific graph properties preserved by coarse graphs, how to measure the differences between original and coarse graphs remains a key challenge In this work, we introduce a new perspective regarding the graph coarsening based on concepts from spectral graph theory We propose and justify new distance functions that characterize the differences between original and coarse graphs We show that the proposed spectral distance naturally captures the structural differences in the graph coarsening process In addition, we provide efficient graph coarsening algorithms to generate graphs which provably preserve the spectral properties from original graphs Experiments show that our proposed algorithms consistently achieve better results compared to previous graph coarsening methods on graph classification and block recovery tasks

Persistent spectral based machine learning (PerSpect ML) for drug design

Abstract: In this paper, we propose persistent spectral based machine learning (PerSpect ML) models for drug design. Persistent spectral models, including persistent spectral graph, persistent spectral simplicial complex and persistent spectral hypergraph, are proposed based on spectral graph theory, spectral simplicial complex theory and spectral hypergraph theory, respectively. Different from all previous spectral models, a filtration process, as proposed in persistent homology, is introduced to generate multiscale spectral models. More specifically, from the filtration process, a series of nested topological representations, i,e., graphs, simplicial complexes, and hypergraphs, can be systematically generated and their spectral information can be obtained. Persistent spectral variables are defined as the function of spectral variables over the filtration value. Mathematically, persistent multiplicity (of zero eigenvalues) is exactly the persistent Betti number (or Betti curve). We consider 11 persistent spectral variables and use them as the feature for machine learning models in protein-ligand binding affinity prediction. We systematically test our models on three most commonly-used databases, including PDBbind-2007, PDBbind-2013 and PDBbind-2016. Our results, for all these databases, are better than all existing models, as far as we know. This demonstrates the great power of our PerSpect ML in molecular data analysis and drug design.

Random walks on a tree with applications.

Abstract: The various dynamics of many complex systems, both natural and artificial, can be well described as random walks on a newly proposed yet powerful object known as a complex network. Here we consider random walks on a special kind of network, a tree. Using the methods of probability-generating functions, we derive a close connection between the widely studied parameters mean first-passage time M and mean shortest path length A on trees. This suggests that for a given tree, the analytic value for M can be easily obtained by calculating the value A when determining the latter is more convenient and vice versa. As a result, the well-known T graph is selected as one of various applications of our methods, and we then obtain an exact solution to its quantity M. On the one hand, the result addressed here is in perfect agreement with previous ones. On the other hand, our method is easier to manipulate than most preexisting ones, for instance, methods from spectral graph theory, since no complicated techniques are involved.

Change detection in noisy dynamic networks: a spectral embedding approach

Abstract: Change detection in dynamic networks is an important problem in many areas, such as fraud detection, cyber intrusion detection and healthcare monitoring. It is a challenging problem because it involves a time sequence of graphs, each of which is usually very large and sparse with heterogeneous vertex degrees, resulting in a complex, high-dimensional mathematical object. Spectral embedding methods provide an effective way to transform a graph to a lower dimensional latent Euclidean space that preserves the underlying structure of the network. Although change detection methods that use spectral embedding are available, they do not address sparsity and degree heterogeneity that usually occur in noisy real-world graphs and a majority of these methods focus on changes in the behaviour of the overall network. In this paper, we adapt previously developed techniques in spectral graph theory and propose a novel concept of applying Procrustes techniques to embedded points for vertices in a graph to detect changes in entity behaviour. Our spectral embedding approach not only addresses sparsity and degree heterogeneity issues, but also obtains an estimate of the appropriate embedding dimension. We call this method CDP (change detection using Procrustes analysis). We demonstrate the performance of CDP through extensive simulation experiments and a real-world application. CDP successfully detects various types of vertex-based changes including (1) changes in vertex degree, (2) changes in community membership of vertices, and (3) unusual increase or decrease in edge weights between vertices. The change detection performance of CDP is compared with two other baseline methods that employ alternative spectral embedding approaches. In both cases, CDP generally shows superior performance.

Fiedler Regularization: Learning Neural Networks with Graph Sparsity

Abstract: We introduce a novel regularization approach for deep learning that incorporates and respects the underlying graphical structure of the neural network. Existing regularization methods often focus on dropping/penalizing weights in a global manner that ignores the connectivity structure of the neural network. We propose to use the Fiedler value of the neural network's underlying graph as a tool for regularization. We provide theoretical support for this approach via spectral graph theory. We list several useful properties of the Fiedler value that makes it suitable in regularization. We provide an approximate, variational approach for fast computation in practical training of neural networks. We provide bounds on such approximations. We provide an alternative but equivalent formulation of this framework in the form of a structurally weighted L1 penalty, thus linking our approach to sparsity induction. We performed experiments on datasets that compare Fiedler regularization with traditional regularization methods such as dropout and weight decay. Results demonstrate the efficacy of Fiedler regularization.

Imaginary replica analysis of loopy regular random graphs

Abstract: We present an analytical approach for describing spectrally constrained maximum entropy ensembles of finitely connected regular loopy graphs, valid in the regime of weak loop-loop interactions. We derive an expression for the leading two orders of the expected eigenvalue spectrum, through the use of infinitely many replica indices taking imaginary values. We apply the method to models in which the spectral constraint reduces to a soft constraint on the number of triangles, which exhibit `shattering' transitions to phases with extensively many disconnected cliques, to models with controlled numbers of triangles and squares, and to models where the spectral constraint reduces to a count of the number of adjacency matrix eigenvalues in a given interval. Our predictions are supported by MCMC simulations based on edge swaps with nontrivial acceptance probabilities.

Graph-theory treatment of one-dimensional strongly repulsive fermions

Abstract: The authors demonstrate that spectral graph theory can effectively describe strongly repulsive one-dimensional mixtures of ultracold fermions and circumvent the high computational complexity of this many-body system.

An ensemble based on a bi-objective evolutionary spectral algorithm for graph clustering

Abstract: Graph clustering is a challenging pattern recognition problem whose goal is to identify vertex partitions with high intra-group connectivity. This paper investigates a bi-objective problem that maximizes the number of intra-cluster edges of a graph and minimizes the expected number of inter-cluster edges in a random graph with the same degree sequence as the original one. The difference between the two investigated objectives is the definition of the well-known measure of graph clustering quality: the modularity. We introduce a spectral decomposition hybridized with an evolutionary heuristic, called MOSpecG, to approach this bi-objective problem and an ensemble strategy to consolidate the solutions found by MOSpecG into a final robust partition. The results of computational experiments with real and artificial LFR networks demonstrated a significant improvement in the results and performance of the introduced method in regard to another bi-objective algorithm found in the literature. The crossover operator based on the geometric interpretation of the modularity maximization problem to match the communities of a pair of individuals was of utmost importance for the good performance of MOSpecG. Hybridizing spectral graph theory and intelligent systems allowed us to define significantly high-quality community structures.

Spectral graph theory efficiently characterizes ventilation heterogeneity in lung airway networks

Abstract: This paper introduces a linear operator for the purposes of quantifying the spectral properties of transport within resistive trees, such as airflow in lung airway networks. The operator, which we ...

Optimizing Synchronizability of Multilayer Networks Based on the Graph Comparison Method

Abstract: This paper is aimed at optimizing the synchronizability of a complex network when the total of its edge weights is given and fixed. We try to allocate edge weights on a complex network to optimize the network's synchronizability from the perspective of spectral graph theory. Most of the existing analysis on multilayer networks assumes the weights of intralayer or interlayer edges to be identical. Such a restrictive assumption is not made in this work. Using the graph comparison based method, different edge weights are allocated according to topological features of networks, which is more reasonable and consistent with most physical complex networks. Furthermore, in order to find out the best edge-weight allocation scheme, we carried out numerical simulations on typical duplex networks and real-world networks. The simulation results show that our proposed edge-weight allocation schemes outperform the average, degree-based, and edge betweenness centrality allocations.

GFCN: A New Graph Convolutional Network Based on Parallel Flows

Abstract: In view of the huge success of convolution neural networks (CNN) for image classification and object recognition, there have been attempts to generalize the method to general graph-structured data. One major direction is based on spectral graph theory. In this paper, we study the problem from a different perspective, by introducing parallel flow decomposition of graphs. The essential idea is to decompose a graph into families of non-intersecting one dimensional (1D) paths, after which, we may apply a 1D CNN along each family of paths. We demonstrate that the our method, which we call GFCN (graph flow convolutional network), is able to transfer CNN architectures to general graphs. We demonstrate effectiveness of the method with synthetic and real applications.

Generalized spectral characterizations of almost controllable graphs

Abstract: Characterizing graphs by their spectra is an important topic in spectral graph theory, which has attracted a lot of attention of researchers in recent years. It is generally very hard and challenging to show a given graph to be determined by its spectrum. In Wang~[J. Combin. Theory, Ser. B, 122 (2017):438-451], the author gave a simple arithmetic condition for a family of graphs being determined by their generalized spectra. However, the method applies only to a family of the so called \emph{controllable graphs}; it fails when the graphs are non-controllable. In this paper, we introduce a class of non-controllable graphs, called \emph{almost controllable graphs}, and prove that, for any pair of almost controllable graphs $G$ and $H$ that are generalized cospectral, there exist exactly two rational orthogonal matrices $Q$ with constant row sums such that $Q^{\rm T}A(G)Q=A(H)$, where $A(G)$ and $A(H)$ are the adjacency matrices of $G$ and $H$, respectively. The main ingredient of the proof is a use of the Binet-Cauchy formula. As an application, we obtain a simple criterion for an almost controllable graph $G$ to be determined by its generalized spectrum, which in some sense extends the corresponding result for controllable graphs.

Option Discovery in the Absence of Rewards with Manifold Analysis

Abstract: Options have been shown to be an effective tool in reinforcement learning, facilitating improved exploration and learning. In this paper, we present an approach based on spectral graph theory and derive an algorithm that systematically discovers options without access to a specific reward or task assignment. As opposed to the common practice used in previous methods, our algorithm makes full use of the spectrum of the graph Laplacian. Incorporating modes associated with higher graph frequencies unravels domain subtleties, which are shown to be useful for option discovery. Using geometric and manifold-based analysis, we present a theoretical justification for the algorithm. In addition, we showcase its performance in several domains, demonstrating clear improvements compared to competing methods.

Local hypergraph clustering using capacity releasing diffusion

Abstract: Local graph clustering is an important machine learning task that aims to find a well-connected cluster near a set of seed nodes. Recent results have revealed that incorporating higher order information significantly enhances the results of graph clustering techniques. The majority of existing research in this area focuses on spectral graph theory-based techniques. However, an alternative perspective on local graph clustering arises from using max-flow and min-cut on the objectives, which offer distinctly different guarantees. For instance, a new method called capacity releasing diffusion (CRD) was recently proposed and shown to preserve local structure around the seeds better than spectral methods. The method was also the first local clustering technique that is not subject to the quadratic Cheeger inequality by assuming a good cluster near the seed nodes. In this paper, we propose a local hypergraph clustering technique called hypergraph CRD (HG-CRD) by extending the CRD process to cluster based on higher order patterns, encoded as hyperedges of a hypergraph. Moreover, we theoretically show that HG-CRD gives results about a quantity called motif conductance, rather than a biased version used in previous experiments. Experimental results on synthetic datasets and real world graphs show that HG-CRD enhances the clustering quality.

The Spectral Zoo of Networks: Embedding and Visualizing Networks with Spectral Moments

Abstract: Network embedding methods have been widely and successfully used in network-based applications such as node classification and link prediction. However, an ideal network embedding should not only be useful for machine learning, but interpretable. We introduce a spectral embedding method for a network, its Spectral Point, which is basically the first few spectral moments of a network. Spectral moments are interpretable, where we prove their close relationships to network structure (e.g. number of triangles and squares) and various network properties (e.g. degree distribution, clustering coefficient, and network connectivity). Using spectral points, we introduce a visualizable and bounded 3D embedding space for all possible graphs, in which one can characterize various types of graphs (e.g., cycles), or real-world networks from different categories (e.g., social or biological networks). We demonstrate that spectral points can be used for network identification (i.e., what network is this subgraph sampled from?) and that by using just the first few moments one does not lose much predictive power.