A major consideration we had in writing this survey was to make it accessible to mathematicians as well as to computer scientists, since expander graphs, the protagonists of our story, come up in numerous and often surprising contexts in both fields But, perhaps, we should start with a few words about graphs in general They are, of course, one of the prime objects of study in Discrete Mathematics However, graphs are among the most ubiquitous models of both natural and human-made structures In the natural and social sciences they model relations among species, societies, companies, etc In computer science, they represent networks of communication, data organization, computational devices as well as the flow of computation, and more In mathematics, Cayley graphs are useful in Group Theory Graphs carry a natural metric and are therefore useful in Geometry, and though they are “just” one-dimensional complexes, they are useful in certain parts of Topology, eg Knot Theory In statistical physics, graphs can represent local connections between interacting parts of a system, as well as the dynamics of a physical process on such systems The study of these models calls, then, for the comprehension of the significant structural properties of the relevant graphs But are there nontrivial structural properties which are universally important? Expansion of a graph requires that it is simultaneously sparse and highly connected Expander graphs were first defined by Bassalygo and Pinsker, and their existence first proved by Pinsker in the early ’70s The property of being an expander seems significant in many of these mathematical, computational and physical contexts It is not surprising that expanders are useful in the design and analysis of communication networks What is less obvious is that expanders have surprising utility in other computational settings such as in the theory of error correcting codes and the theory of pseudorandomness In mathematics, we will encounter eg their role in the study of metric embeddings, and in particular in work around the Baum-Connes Conjecture Expansion is closely related to the convergence rates of Markov Chains, and so they play a key role in the study of Monte-Carlo algorithms in statistical mechanics and in a host of practical computational applications The list of such interesting and fruitful connections goes on and on with so many applications we will not even

/pdf/expander-graphs-and-their-applications-wbsb8gxfw2.pdf

Expander graphs and their applications

In this paper, we consider the question of determining whether a function f has property P or is e-far from any function with property P. A property testing algorithm is given a sample of the value of f on instances drawn according to some distribution. In some cases, it is also allowed to query f on instances of its choice. We study this question for different properties and establish some connections to problems in learning theory and approximation.In particular, we focus our attention on testing graph properties. Given access to a graph G in the form of being able to query whether an edge exists or not between a pair of vertices, we devise algorithms to test whether the underlying graph has properties such as being bipartite, k-Colorable, or having a p-Clique (clique of density p with respect to the vertex set). Our graph property testing algorithms are probabilistic and make assertions that are correct with high probability, while making a number of queries that is independent of the size of the graph. Moreover, the property testing algorithms can be used to efficiently (i.e., in time linear in the number of vertices) construct partitions of the graph that correspond to the property being tested, if it holds for the input graph.

Property Testing and its connection to Learning and Approximation

Recently several results appeared that show significant reduction in time for matrix multiplication, singular value decomposition as well as linear (\ell_ 2) regression, all based on data dependent random sampling. Our key idea is that low dimensional embeddings can be used to eliminate data dependence and provide more versatile, linear time pass efficient matrix computation. Our main contribution is summarized as follows. --Independent of the recent results of Har-Peled and of Deshpande and Vempala, one of the first -- and to the best of our knowledge the most efficient -- relative error (1 + \in) \parallel A - A_k \parallel _F approximation algorithms for the singular value decomposition of an m ? n matrix A with M non-zero entries that requires 2 passes over the data and runs in time O\left( {\left( {M(\frac{k} { \in } + k\log k) + (n + m)(\frac{k} { \in } + k\log k)^2 } \right)\log \frac{1} {\delta }} \right) --The first o(nd^{2}) time (1 + \in) relative error approximation algorithm for n ? d linear (\ell_2) regression. --A matrix multiplication and norm approximation algorithm that easily applies to implicitly given matrices and can be used as a black box probability boosting tool.

Improved Approximation Algorithms for Large Matrices via Random Projections

Starting around the late 1950s, several research communities began relating the geometry of graphs to stochastic processes on these graphs. This book, twenty years in the making, ties together research in the field, encompassing work on percolation, isoperimetric inequalities, eigenvalues, transition probabilities, and random walks. Written by two leading researchers, the text emphasizes intuition, while giving complete proofs and more than 850 exercises. Many recent developments, in which the authors have played a leading role, are discussed, including percolation on trees and Cayley graphs, uniform spanning forests, the mass-transport technique, and connections on random walks on graphs to embedding in Hilbert space. This state-of-the-art account of probability on networks will be indispensable for graduate students and researchers alike.

Probability on Trees and Networks

Determinantal point processes (DPPs) are elegant probabilistic models of repulsion that arise in quantum physics and random matrix theory. In contrast to traditional structured models like Markov random fields, which become intractable and hard to approximate in the presence of negative correlations, DPPs offer efficient and exact algorithms for sampling, marginalization, conditioning, and other inference tasks. We provide a gentle introduction to DPPs, focusing on the intuitions, algorithms, and extensions that are most relevant to the machine learning community, and show how DPPs can be applied to real-world applications like finding diverse sets of high-quality search results, building informative summaries by selecting diverse sentences from documents, modeling non-overlapping human poses in images or video, and automatically building timelines of important news stories.

/pdf/determinantal-point-processes-for-machine-learning-4jsyx0d5k9.pdf

Determinantal point processes for machine learning

We show how to determine whether the edit distance between two given strings is small in sublinear time. Specifically, we present a test which, given two n-character strings A and B, runs in time o(n) and with high probability returns "CLOSE" if their edit distance is O(nΑ), and "FAR" if their edit distance is Ω(n), where Α is a fixed parameter less than 1. Our algorithm for testing the edit distance works by recursively subdividing the strings A and B into smaller substrings and looking for pairs of substrings in A, B with small edit distance. To do this, we query both strings at random places using a special technique for economizing on the samples which does not pick the samples independently and provides better query and overall complexity. As a result, our test runs in time O(nmax(Α/2, 2Α - 1\)) for any fixed Α

/pdf/a-sublinear-algorithm-for-weakly-approximating-edit-distance-3pwn1mzolu.pdf

A sublinear algorithm for weakly approximating edit distance

We present a new method for proving rank lower bounds for Cutting Planes (CP) and several procedures based on lifting due to Lovasz and Schrijver (LS), when viewed as proof systems for unsatisfiability We apply this method to obtain the following new results: first, we prove near-optimal rank bounds for Cutting Planes and Lovasz-Schrijver proofs for several prominent unsatisfiable CNF examples, including random kCNF formulas and the Tseitin graph formulas It follows from these lower bounds that a linear number of rounds of CP or LS procedures when applied to relaxations of integer linear programs is not sufficient for reducing the integrality gap Secondly, we give unsatisfiable examples that have constant rank CP and LS proofs but that require linear rank resolution proofs Thirdly, we give examples where the CP rank is O(log n) but the LS rank is linear Finally, we address the question of size versus rank: we show that, for both proof systems, rank does not accurately reflect proof size Specifically, there are examples with polynomial-size CP/LS proofs, but requiring linear rank

/pdf/rank-bounds-and-integrality-gaps-for-cutting-planes-3ynxiu7eck.pdf

Rank Bounds and Integrality Gaps for Cutting Planes Procedures

Let X be a subset of n points of the Euclidean space, and let 0 < ɛ < 1. A classical result of Johnson and Lindenstrauss [JL84] states that there is a projection of X onto a subspace of dimension O(ɛ-2 log n), with distortion ≤ 1 + ɛ. Here we show a natural extension of the above result, to a stronger preservation of the geometry of finite spaces. By a k-fold increase of the number of dimensions used compared to [JL84], a good preservation of volumes and of distances between points and affine spaces is achieved. Specifically, we show it is possible to embed a subset of size n of the Euclidean space into a O(ɛ-2 klogn)-dimensional Euclidean space, so that no set of size s ≤ k changes its volume by more than (1+ɛ)s-1. Moreover, distances of points from affine hulls of sets of at most k-1 points in the space do not change by more than a factor of 1 + ɛ. A consequence of the above with k = 3 is that angles can be preserved using asymptotically the same number of dimensions as the one used in [JL84]. Our method can be applied to many problems with high-dimensional nature such as Projective Clustering and Approximated Nearest Affine Neighbor Search. In particular, it shows a first poly-logarithmic query time approximation algorithm to the latter. We also show a structural application that for volume respecting embedding in the sense introduced by Feige [Fei00], the host space need not generally be of dimensionality greater than polylogarithmic in the size of the graph.

Dimensionality Reductions That Preserve Volumes and Distance to Affine Spaces, and Their Algorithmic Applications

We consider the problem of approximating fixed-predicate constraint satisfaction problems (MAX k-CSPq(P)), where the variables take values from (q) =f0; 1;:::; q 1g, and each constraint is on k variables and is defined by a fixed k-ary predicate P. Familiar problems like MAX 3-SAT and MAX-CUT belong to this category. Austrin and Mossel recently identified a general class of predicates P for which MAX k-CSPq(P) is hard to approximate. They study predicates P : (q) k !f0; 1g such that the set of assignments accepted by P contains the support of a balanced pairwise independent distribution over the domain of the inputs. We refer to such predicates as promising. Austrin and Mossel show that for any promising predicate P, the problem MAX k-CSPq(P) is Unique-Games-hard to approximate better than the trivial approximation obtained by a random assignment. We give an unconditional analogue of this result in a restricted model of computation. We consider the hierarchy of semidefinite relaxations of MAX k-CSPq(P) obtained by augmenting the canonical semidefinite relaxation with the Sherali-Adams hierarchy. We show that for any promising predicate P, the integrality gap remains the same as the approximation ratio achieved by a random assignment, even after W(n) levels of this hierarchy.

/pdf/sdp-gaps-from-pairwise-independence-2zntjznni4.pdf

SDP Gaps from Pairwise Independence

We consider the problem of computing the weight of a Euclidean minimum spanning tree for a set of n points in $\mathbb R^d$. We focus on the setting where the input point set is supported by certain basic (and commonly used) geometric data structures that can provide efficient access to the input in a structured way. We present an algorithm that estimates with high probability the weight of a Euclidean minimum spanning tree of a set of points to within $1 + \eps$ using only $\widetilde{\O}(\sqrt{n} \, \text{poly} (1/\eps))$ queries for constant d. The algorithm assumes that the input is supported by a minimal bounding cube enclosing it, by orthogonal range queries, and by cone approximate nearest neighbor queries.

Avner Magen

Papers

A sublinear algorithm for weakly approximating edit distance

Rank Bounds and Integrality Gaps for Cutting Planes Procedures

Dimensionality Reductions That Preserve Volumes and Distance to Affine Spaces, and Their Algorithmic Applications

SDP Gaps from Pairwise Independence

Approximating the Weight of the Euclidean Minimum Spanning Tree in Sublinear Time