Graph sketching is a powerful technique introduced by the seminal work of Ahn, Guha and McGregor'12 on connectivity in dynamic graph streams that has enjoyed considerable attention in the literature since then, and has led to near optimal dynamic streaming algorithms for many fundamental problems such as connectivity, cut and spectral sparsifiers and matchings. Interestingly, however, the sketching and dynamic streaming complexity of approximating the shortest path metric of a graph is still far from well-understood. Besides a direct $k$-pass implementation of classical spanner constructions (recently improved to $\lfloor\frac k2\rfloor+1$-passes by Fernandez, Woodruff and Yasuda'20) the state of the art amounts to a $O(\log k)$-pass algorithm of Ahn, Guha and McGregor'12, and a $2$-pass algorithm of Kapralov and Woodruff'14. In particular, no single pass algorithm is known, and the optimal tradeoff between the number of passes, stretch and space complexity is open. 
In this paper we introduce several new graph sketching techniques for approximating the shortest path metric of the input graph. We give the first {\em single pass} sketching algorithm for constructing graph spanners: we show how to obtain a $\widetilde{O}(n^{\frac23})$-spanner using $\widetilde{O}(n)$ space, and in general a $\widetilde{O}(n^{\frac23(1-\alpha)})$-spanner using $\widetilde{O}(n^{1+\alpha})$ space for every $\alpha\in [0, 1]$, a tradeoff that we think may be close optimal. We also give new spanner construction algorithms for any number of passes, simultaneously improving upon all prior work on this problem. Finally, we study the simultaneous communication model and propose the first protocols with low per player information.

Graph Spanners by Sketching in Dynamic Streams and the Simultaneous Communication Model

We introduce a new measure for quantifying the amount of information that the nodes in a network need to learn to jointly solve a graph problem. We show that the local information cost ($\textsf{LIC}$) presents a natural lower bound on the communication complexity of distributed algorithms. For the synchronous CONGEST-KT1 model, where each node has initial knowledge of its neighbors' IDs, we prove that $\Omega(\textsf{LIC}_\gamma(P)/ \log\tau \log n)$ bits are required for solving a graph problem $P$ with a $\tau$-round algorithm that errs with probability at most $\gamma$. Our result is the first lower bound that yields a general trade-off between communication and time for graph problems in the CONGEST-KT1 model. 
We demonstrate how to apply the local information cost by deriving a lower bound on the communication complexity of computing a $(2t-1)$-spanner that consists of at most $O(n^{1+1/t + \epsilon})$ edges, where $\epsilon = \Theta(1/t^2)$. Our main result is that any $O(\textsf{poly}(n))$-time algorithm must send at least $\tilde\Omega((1/t^2) n^{1+1/2t})$ bits in the CONGEST model under the KT1 assumption. Previously, only a trivial lower bound of $\tilde \Omega(n)$ bits was known for this problem. 
A consequence of our lower bound is that achieving both time- and communication-optimality is impossible when designing a distributed spanner algorithm. In light of the work of King, Kutten, and Thorup (PODC 2015), this shows that computing a minimum spanning tree can be done significantly faster than finding a spanner when considering algorithms with $\tilde O(n)$ communication complexity. Our result also implies time complexity lower bounds for constructing a spanner in the node-congested clique of Augustine et al. (2019) and in the push-pull gossip model with limited bandwidth.

Being Fast Means Being Chatty: The Local Information Cost of Graph Spanners

A classic 1993 paper by Alth\H{o}fer et al. proved a tight reduction from spanners, emulators, and distance oracles to the extremal function $\gamma$ of high-girth graphs. This paper initiated a large body of work in network design, in which problems are attacked by reduction to $\gamma$ or the analogous extremal function for other girth concepts. In this paper, we introduce and study a new girth concept that we call the bridge girth of path systems, and we show that it can be used to significantly expand and improve this web of connections between girth problems and network design. We prove two kinds of results: 1) We write the maximum possible size of an $n$-node, $p$-path system with bridge girth $>k$ as $\beta(n, p, k)$, and we write a certain variant for"ordered"path systems as $\beta^*(n, p, k)$. We identify several arguments in the literature that implicitly show upper or lower bounds on $\beta, \beta^*$, and we provide some polynomially improvements to these bounds. In particular, we construct a tight lower bound for $\beta(n, p, 2)$, and we polynomially improve the upper bounds for $\beta(n, p, 4)$ and $\beta^*(n, p, \infty)$. 2) We show that many state-of-the-art results in network design can be recovered or improved via black-box reductions to $\beta$ or $\beta^*$. Examples include bounds for distance/reachability preservers, exact hopsets, shortcut sets, the flow-cut gaps for directed multicut and sparsest cut, an integrality gap for directed Steiner forest. We believe that the concept of bridge girth can lead to a stronger and more organized map of the research area. Towards this, we leave many open problems, related to both bridge girth reductions and extremal bounds on the size of path systems with high bridge girth.

Bridge Girth: A Unifying Notion in Network Design

Scalable kernel methods, including kernel ridge regression, often rely on low-rank matrix approximations using the Nystrom method, which involves selecting landmark points from large data sets. The existing approaches to selecting landmarks are typically computationally demanding as they require manipulating and performing computations with large matrices in the input or feature space. In this paper, our contribution is twofold. The first contribution is to propose a novel landmark selection method that promotes diversity using an efficient two-step approach. Our landmark selection technique follows a coarse to fine strategy, where the first step computes importance scores with a single pass over the whole data. The second step performs K-means clustering on the constructed coreset to use the obtained centroids as landmarks. Hence, the introduced method provides tunable trade-offs between accuracy and efficiency. Our second contribution is to investigate the performance of several landmark selection techniques using a novel application of kernel methods for predicting structural responses due to earthquake load and material uncertainties. Our experiments exhibit the merits of our proposed landmark selection scheme against baselines.

Kernel Ridge Regression Using Importance Sampling with Application to Seismic Response Prediction

We present a sublinear query algorithm for outputting a near-optimal low-rank approximation to any positive semidefinite Toeplitz matrix $T \in \mathbb{R}^{d \times d}$. In particular, for any integer rank $k \leq d$ and $\epsilon,\delta>0$, our algorithm makes $\tilde{O} \left (k^2 \cdot \log(1/\delta) \cdot \text{poly}(1/\epsilon) \right )$ queries to the entries of $T$ and outputs a rank $\tilde{O} \left (k \cdot \log(1/\delta)/\epsilon\right )$ matrix $\tilde{T} \in \mathbb{R}^{d \times d}$ such that $\| T - \tilde{T}\|_F \leq (1+\epsilon) \cdot \|T-T_k\|_F + \delta \|T\|_F$. Here, $\|\cdot\|_F$ is the Frobenius norm and $T_k$ is the optimal rank-$k$ approximation to $T$, given by projection onto its top $k$ eigenvectors. $\tilde{O}(\cdot)$ hides $\text{polylog}(d) $ factors. Our algorithm is \emph{structure-preserving}, in that the approximation $\tilde{T}$ is also Toeplitz. A key technical contribution is a proof that any positive semidefinite Toeplitz matrix in fact has a near-optimal low-rank approximation which is itself Toeplitz. Surprisingly, this basic existence result was not previously known. Building on this result, along with the well-established off-grid Fourier structure of Toeplitz matrices [Cybenko'82], we show that Toeplitz $\tilde{T}$ with near optimal error can be recovered with a small number of random queries via a leverage-score-based off-grid sparse Fourier sampling scheme.

Toeplitz Low-Rank Approximation with Sublinear Query Complexity

Graph spanners are sparse subgraphs which approximately preserve all pairwise shortest-path distances in an input graph. The notion of approximation can be additive, multiplicative, or both, and many variants of this problem have been extensively studied. We study the problem of computing a graph spanner when the edges of the input graph are distributed across two or more sites in an arbitrary, possibly worst-case partition, and the goal is for the sites to minimize the communication used to output a spanner. We assume the message-passing model of communication, for which there is a point-to-point link between all pairs of sites as well as a coordinator who is responsible for producing the output. We stress that the subset of edges that each site has is not related to the network topology, which is fixed to be point-to-point. While this model has been extensively studied for related problems such as graph connectivity, it has not been systematically studied for graph spanners. We present the first tradeoffs for total communication versus the quality of the spanners computed, for two or more sites, as well as for additive and multiplicative notions of distortion. We show separations in the communication complexity when edges are allowed to occur on multiple sites, versus when each edge occurs on at most one site. We obtain nearly tight bounds (up to polylog factors) for the communication of additive $2$-spanners in both the with and without duplication models, multiplicative $(2k-1)$-spanners in the with duplication model, and multiplicative $3$ and $5$-spanners in the without duplication model. Our lower bound for multiplicative $3$-spanners employs biregular bipartite graphs rather than the usual Erdős girth conjecture graphs and may be of wider interest.

Graph Spanners in the Message-Passing Model

/pdf/tight-kernel-query-complexity-of-kernel-ridge-regression-and-40bt8qdi1q.pdf

Tight Kernel Query Complexity of Kernel Ridge Regression and Kernel $k$-means Clustering.

We present tight lower bounds on the number of kernel evaluations required to approximately solve kernel ridge regression (KRR) and kernel $k$-means clustering (KKMC) on $n$ input points. For KRR, our bound for relative error approximation to the minimizer of the objective function is $\Omega(nd_{\mathrm{eff}}^\lambda/\varepsilon)$ where $d_{\mathrm{eff}}^\lambda$ is the effective statistical dimension, which is tight up to a $\log(d_{\mathrm{eff}}^\lambda/\varepsilon)$ factor. For KKMC, our bound for finding a $k$-clustering achieving a relative error approximation of the objective function is $\Omega(nk/\varepsilon)$, which is tight up to a $\log(k/\varepsilon)$ factor. Our KRR result resolves a variant of an open question of El Alaoui and Mahoney, asking whether the effective statistical dimension is a lower bound on the sampling complexity or not. Furthermore, for the important practical case when the input is a mixture of Gaussians, we provide a KKMC algorithm which bypasses the above lower bound.

Taisuke Yasuda

Papers

Graph Spanners in the Message-Passing Model

Tight Kernel Query Complexity of Kernel Ridge Regression and Kernel $k$-means Clustering.

Tight Kernel Query Complexity of Kernel Ridge Regression and Kernel $k$-means Clustering

Graph Spanners in the Message-Passing Model.

The Query Complexity of Mastermind with l p Distances.