Communication complexity

About: Communication complexity is a research topic. Over the lifetime, 3870 publications have been published within this topic receiving 105832 citations.

The unbounded-error communication complexity of symmetric functions

Abstract: We prove an essentially tight lower bound on the unbounded-error communication complexity of every symmetric function, ie, f(x,y)=D(|x∧y|), where D: {0,1,…,n}→{0,1} is a given predicate and x,y range over {0,1}n Specifically, we show that the communication complexity of f is between Θ(k/log5 n) and Θ(k logn), where k is the number of value changes of D in {0,1,…, n} Prior to this work, the problem was solved only for the parity predicate D (Forster 2001) Our proof is built around two new ideas First, we show that a predicate D gives rise to a rapidly mixing random walk on ℤ2 n , which allows us to reduce the problem to communication lower bounds for “typical” predicates Second, we use Paturi’s approximation lower bounds (1992), suitably generalized here to clusters of real nodes in [0,n] and interpreted in their dual form, to prove that a typical predicate behaves analogous to the parity predicate with respect to a smooth distribution on the inputs

A Better Alternative to Error Feedback for Communication-Efficient Distributed Learning

Abstract: Modern large-scale machine learning applications require stochastic optimization algorithms to be implemented on distributed compute systems. A key bottleneck of such systems is the communication overhead for exchanging information across the workers, such as stochastic gradients. Among the many techniques proposed to remedy this issue, one of the most successful is the framework of compressed communication with error feedback (EF). EF remains the only known technique that can deal with the error induced by contractive compressors which are not unbiased, such as Top-$K$. In this paper, we propose a new and theoretically and practically better alternative to EF for dealing with contractive compressors. In particular, we propose a construction which can transform any contractive compressor into an induced unbiased compressor. Following this transformation, existing methods able to work with unbiased compressors can be applied. We show that our approach leads to vast improvements over EF, including reduced memory requirements, better communication complexity guarantees and fewer assumptions. We further extend our results to federated learning with partial participation following an arbitrary distribution over the nodes, and demonstrate the benefits thereof. We perform several numerical experiments which validate our theoretical findings.

A Distributed Geometric Routing Algorithm for Ad HocWireless Networks

Abstract: In this paper, we presented a fully distributed algorithm to compute a planar subgraph of the underlying wireless connectivity graph. We considered the idealized unit disk graph model in which nodes are assumed to be connected if and only if nodes are within their transmission range. The main contribution of this work is a fully distributed algorithm to extract the connected, planar graph for routing in the wireless networks. The communication cost of the proposed algorithm is O(d log d) bits, where d is the degree of anode. In addition, this paper also presented a geometric routing algorithm. The algorithm is fully distributed and nodes know only the position of other nodes and can communicate with neighboring nodes in their transmission range

Bandwidth Efficient String Reconciliation Using Puzzles

Abstract: Of considerable interest in recent years has been the problem of exchanging correlated data with minimum communication. We thus consider the problem of exchanging two similar strings held by different hosts. Our approach involves transforming a string into a multiset of substrings that are reconciled efficiently using known multiset reconciliation algorithms, and then put back together on a remote host using tools from graph theory. We present analyses, experiments, and results to show that the communication complexity of our approach for high-entropy data compares favorably to existing algorithms including rsync, a widely-used string reconciliation engine. We also quantify the trade-off between communication and the computation complexity of our approach

Low-complexity multiuser detection and reduced-rank Wiener filters for ultra-wideband multiple access

Abstract: Realizing the large user capacity planned for ultra-wideband (UWB) systems motivates multiuser detection (MUD). However, it is impractical to implement conventional chip-rate MUD methods, because UWB signaling gives rise to high detection complexity and difficulty in capturing energy scattered by dense multipath. In this paper, we develop a reception model for UWB multiple access based on frame-rate sampled signals in lieu of chip-rate samples. This model enables low-complexity MUD, of which we examine a reduced-rank Wiener filter for blind symbol detection. We show that frame-rate UWB samples have a small number of distinct eigenvalues in the data covariance matrix, resulting in warp convergence of reduced-rank filtering. The proposed MUD method exhibits good performance at low complexity, even in the presence of strong frequency-selective multipath fading.