Showing papers on "Communication complexity published in 2021"

Federated Learning of a Mixture of Global and Local Models

Abstract: We propose a new optimization formulation for training federated learning models. The standard formulation has the form of an empirical risk minimization problem constructed to find a single global model trained from the private data stored across all participating devices. In contrast, our formulation seeks an explicit trade-off between this traditional global model and the local models, which can be learned by each device from its own private data without any communication. Further, we develop several efficient variants of SGD (with and without partial participation and with and without variance reduction) for solving the new formulation and prove communication complexity guarantees. Notably, our methods are similar but not identical to federated averaging / local SGD, thus shedding some light on the essence of the elusive method. In particular, our methods do not perform full averaging steps and instead merely take steps towards averaging. We argue for the benefits of this new paradigm for federated learning.

A Scalable Multi-Layer PBFT Consensus for Blockchain

Abstract: Practical Byzantine Fault Tolerance (PBFT) consensus mechanism shows a great potential to break the performance bottleneck of the Proof-of-Work (PoW)-based blockchain systems, which typically support only dozens of transactions per second and require minutes to hours for transaction confirmation. However, due to frequent inter-node communications, PBFT mechanism has a poor node scalability and thus it is typically adopted in small networks. To enable PBFT in large systems such as massive Internet of Things (IoT) ecosystems and blockchain, in this article, a scalable multi-layer PBFT-based consensus mechanism is proposed by hierarchically grouping nodes into different layers and limiting the communication within the group. We first propose an optimal double-layer PBFT and show that the communication complexity is significantly reduced. Specifically, we prove that when the nodes are evenly distributed within the sub-groups in the second layer, the communication complexity is minimized. The security threshold is analyzed based on faulty probability determined (FPD) and faulty number determined (FND) models, respectively. We also provide a practical protocol for the proposed double-layer PBFT system. Finally, the results are extended to arbitrary-layer PBFT systems with communication complexity and security analysis. Simulation results verify the effectiveness of the analytical results.

Wolverine: Fast, Scalable, and Communication-Efficient Zero-Knowledge Proofs for Boolean and Arithmetic Circuits

Abstract: Efficient zero-knowledge (ZK) proofs for arbitrary boolean or arithmetic circuits have recently attracted much attention. Existing solutions suffer from either significant prover overhead (i.e., high memory usage) or relatively high communication complexity (at least κ bits per gate, for computational security parameter κ). In this paper, we propose a new protocol for constant-round interactive ZK proofs that simultaneously allows for an efficient prover with asymptotically optimal memory usage and significantly lower communication compared to protocols with similar memory efficiency. Specifically:•The prover in our ZK protocol has linear running time and, perhaps more importantly, memory usage linear in the memory needed to evaluate the circuit non-cryptographically. This allows our proof system to scale easily to very large circuits.•for statistical security parameter ρ = 40, our ZK protocol communicates roughly 9 bits/gate for boolean circuits and 2–4 field elements/gate for arithmetic circuits over large fields.Using 5 threads, 400 MB of memory, and a 200 Mbps network to evaluate a circuit with hundreds of billions of gates, our implementation (ρ = 40, κ = 128) runs at a rate of 0.45 μs/gate in the boolean case, and 1.6 μs/gate for an arithmetic circuit over a 61-bit field.We also present an improved subfield Vector Oblivious Linear Evaluation (sVOLE) protocol with malicious security that is of independent interest.

All You Need is DAG

Abstract: We present DAG-Rider, the first asynchronous Byzantine Atomic Broadcast protocol that achieves optimal resilience, optimal amortized communication complexity, and optimal time complexity. DAG-Rider is post-quantum safe and ensures that all values proposed by correct processes eventually get delivered. We construct DAG-Rider in two layers: In the first layer, processes reliably broadcast their proposals and build a structured Directed Acyclic Graph (DAG) of the communication among them. In the second layer, processes locally observe their DAGs and totally order all proposals with no extra communication.

Asynchronous Data Dissemination and its Applications

Abstract: In this paper, we introduce the problem of Asynchronous Data Dissemination (ADD). Intuitively, an ADD protocol disseminates a message to all honest nodes in an asynchronous network, given that at least t+1 honest nodes initially hold the message where t is the maximum number of malicious nodes. We design a simple and efficient ADD protocol for n parties that is information-theoretically secure, tolerates up to one-third malicious nodes, and has a communication cost of O(n|M|+n2) for disseminating a message M. We then use our ADD protocol to improve many important primitives in cryptography and distributed computing. For asynchronous reliable broadcast (RBC), assuming collision-resistant hash functions, we give a RBC protocol with communication cost O(n|M| + κ n2) where κ is the size of the hash function output. This improves over the prior best scheme with communication cost O(n|M| + κ n2 log n) under the same setting. Our improved RBC protocol immediately improves the communication cost of asynchronous atomic broadcast and Asynchronous Distributed Key Generation (ADKG) protocols. We also use our improved RBC protocol along with additional new techniques to improve the communication cost of Asynchronous Verifiable Secret Sharing (AVSS), Asynchronous Complete Secret Sharing (ACSS), and dual-threshold ACSS from O(κ n2 log n) to O(κ n2) without using any trusted setup.

FedPD: A Federated Learning Framework With Adaptivity to Non-IID Data

Abstract: Federated Learning (FL) is popular for communication-efficient learning from distributed data. To utilize data at different clients without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a computation then aggregation model, in which multiple local updates are performed using local data before aggregation. These algorithms fail to work when faced with practical challenges, e.g., the local data being non-identically independently distributed. In this paper, we first characterize the behavior of the FedAvg algorithm, and show that without strong and unrealistic assumptions on the problem structure, it can behave erratically. Aiming at designing FL algorithms that are provably fast and require as few assumptions as possible, we propose a new algorithm design strategy from the primal-dual optimization perspective. Our strategy yields algorithms that can deal with non-convex objective functions, achieves the best possible optimization and communication complexity (in a well-defined sense), and accommodates full-batch and mini-batch local computation models. Importantly, the proposed algorithms are communication efficient , in that the communication effort can be reduced when the level of heterogeneity among the local data also reduces. In the extreme case where the local data becomes homogeneous, only $\mathcal {O}(1)$ communication is required among the agents. To the best of our knowledge, this is the first algorithmic framework for FL that achieves all the above properties.

QuickSilver: Efficient and Affordable Zero-Knowledge Proofs for Circuits and Polynomials over Any Field

Abstract: Zero-knowledge (ZK) proofs with an optimal memory footprint have attracted a lot of attention, because such protocols can easily prove very large computation with a small memory requirement. Such ZK protocol only needs O(M) memory for both parties, where M is the memory required to verify the statement in the clear. In this paper, we propose several new constant-round ZK protocols in this setting, which improve the concrete efficiency and, at the same time, enable sublinear amortized communication for circuits with some notion of relaxed uniformity. In the circuit-based model, where the computation is represented as a circuit over a field, our ZK protocol achieves a communication complexity of 1 field element per non-linear gate for any field size while keeping the computation very cheap. We implemented our protocol, which shows extremely high efficiency and affordability. Compared to the previous best-known implementation, we achieve 6x--7x improvement in computation and 3x--7x improvement in communication. When running on intro-level AWS instances, our protocol only needs one US dollar to prove one trillion AND gates (or 2.5 US dollars for one trillion multiplication gates over a 61-bit field). In the setting where part of the computation can be represented as a set of polynomials with a "degree-separated" format, we can achieve communication sublinear to the polynomial size: the communication only depends on the total number of distinct variables in all the polynomials and the highest degree of all polynomials, independent of the number of multiplications to compute all polynomials. Using the improved ZK protocol, we can prove matrix multiplication with communication proportional to the input size, rather than the number of multiplications. Proving the multiplication of two 1024 x 1024 matrices, our implementation, with one thread and 1 GB of memory, only needs 10 seconds and communicates 25 MB.

A Proof-of-Quality-Factor (PoQF)-Based Blockchain and Edge Computing for Vehicular Message Dissemination

Abstract: Blockchain applications in vehicular networks can offer many advantages, including decentralization and improved security. However, most of the consensus algorithms in blockchain are difficult to be implemented in vehicular ad hoc networks (VANETs) without the help of edge computing services. For example, the connectivity in VANET only remains for a short period of time, which is not sufficient for highly time-consuming consensus algorithms, e.g., Proof of Work, running on mobile-edge nodes (vehicles). Other consensus algorithms also have some drawbacks, e.g., Proof of Stake (PoS) is biased toward nodes with a higher amount of stakes and Proof of Elapsed Time (PoET) is not highly secure against malicious nodes. For these reasons, we propose a voting blockchain based on the Proof-of-Quality-Factor (PoQF) consensus algorithm, where the threshold number of votes is controlled by edge computing servers. Specifically, PoQF includes voting for message validation and a competitive relay selection process based on the probabilistic prediction of channel quality between the transmitter and receiver. The performance bounds of failure and latency in message validation are obtained. This article also analyzes the throughput of block generation, as well as the asymptotic latency, security, and communication complexity of PoQF. An incentive distribution mechanism to reward honest nodes and punish malicious nodes is further presented and its effectiveness against the collusion of nodes is proved using the game theory. Simulation results show that PoQF reduces failure in validation by 11% and 15% as compared to PoS and PoET, respectively, and is 68 ms faster than PoET.

Cloud-Based Quadratic Optimization With Partially Homomorphic Encryption

Abstract: This article develops a cloud-based protocol for a constrained quadratic optimization problem involving multiple parties, each holding private data. The protocol is based on the projected gradient ascent on the Lagrange dual problem and exploits partially homomorphic encryption and secure communication techniques. Using formal cryptographic definitions of indistinguishability, the protocol is shown to achieve computational privacy. We show the implementation results of the protocol and discuss its computational and communication complexity. We conclude this article with a discussion on privacy notions.

Labeled PSI from Homomorphic Encryption with Reduced Computation and Communication

Abstract: It is known that fully homomorphic encryption (FHE) can be used to build efficient (labeled) Private Set Intersection protocols in the unbalanced setting, where one of the sets is much larger than the other~(Chen et al. (CCS'17, CCS'18)). In this paper we demonstrate multiple algorithmic improvements upon these works. In particular, our protocol has an asymptotically better computation cost, requiring only O(√|X| ) homomorphic multiplications, and communication complexity sublinear in the larger set size|X|. We demonstrate that our protocol is significantly better than that of Chen et al. (CCS'18) for many practical parameters, especially in terms of online communication cost. For example, when intersecting $228 and 2048 item sets, our protocol reduces the online computation time by more than 71% and communication by more than 63%. When intersecting 224 and 4096 item sets, our protocol reduces the online computation time by 27% and communication by 63%. Our comparison to other state-of-the-art unbalanced PSI protocols shows that our protocol has the best total communication complexity when |X| ≥ 224. For labeled PSI our protocol also outperforms Chen et al. (CCS'18). When intersecting 220 and 256 item sets, with the larger set having associated 288-byte labels, our protocol reduces the online computation time by more than 67% and communication by 34%. Finally, we demonstrate a modification that results in nearly constant communication cost in the larger set size |X|, but impractically high computation complexity on today's CPUs. For example, to intersect a 210-item set with sets of size 222, 224, or 226, our proof-of-concept implementation requires only 0.76 MB of online communication, which is more than a 24-fold improvement over Chen et al. (CCS'18).

E health care data privacy preserving efficient file retrieval from the cloud service provider using attribute based file encryption

Abstract: File storing and retrieving is performed in the robust as well as secure manner by using the cloud computing technology. Various researchers have developed numerous mechanisms via attribute based encryption for the health care applications. Although, more protocols developed among them only very few techniques were efficient and robust for the quick retrieval of reports from the cloud but many protocols suffer by reason of less security, confidentiality and integrity. Existing techniques was based on encrypting the file based on the keyword. But in our proposed protocol, we have developed an attribute based encryption which will overcome the issues faced by the previous research techniques. The group of patient records are encrypted with single common attribute. From the survey, it is clear that the existing protocols suffer due to high computation and communication complexity. So as to rectify the existing issue, we proposed the effective recovery of files by using attribute based file encryption mechanism from cloud (ERFC). When comparing to the existing protocols, our proposed ERFC mechanism takes minimum computation and communication complexity for four working mechanisms namely, patient key computation, doctor index building computation, cloud working mechanism and finally patient report decryption. All these four working mechanisms are developed for effective recovery of files to the end users. Our proposed protocol is secure against some attacks like Eavesdropping, masquerade, replay and man in the middle attack. Our performance analysis section describes that our ERFC mechanism is better with communication as well as computation complexity when related to the other existing protocols.

Multi-party Threshold Private Set Intersection with Sublinear Communication

Abstract: In multi-party threshold private set intersection (PSI), n parties each with a private set wish to compute the intersection of their sets if the intersection is sufficiently large. Previously, Ghosh and Simkin (CRYPTO 2019) studied this problem for the two-party case and demonstrated interesting lower and upper bounds on the communication complexity. In this work, we investigate the communication complexity of the multi-party setting \((n\ge 2)\). We consider two functionalities for multi-party threshold PSI. In the first, parties learn the intersection if each of their sets and the intersection differ by at most T. In the second functionality, parties learn the intersection if the union of all their sets and the intersection differ by at most T.

Distributed Stochastic Consensus Optimization With Momentum for Nonconvex Nonsmooth Problems

Abstract: While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems. Based on a proximal primal-dual approach, this paper presents a new (stochastic) distributed algorithm with Nesterov momentum for accelerated optimization of non-convex and non-smooth problems. Theoretically, we show that the proposed algorithm can achieve an $\epsilon$ -stationary solution under a constant step size with $\mathcal {O}(1/\epsilon ^2)$ computation complexity and $\mathcal {O}(1/\epsilon)$ communication complexity when the epigraph of the non-smooth term is a polyhedral set. When compared to the existing gradient tracking based methods, the proposed algorithm has the same order of computation complexity but lower order of communication complexity. To the best of our knowledge, the presented result is the first stochastic algorithm with the $\mathcal {O}(1/\epsilon)$ communication complexity for non-convex and non-smooth problems. Numerical experiments for a distributed non-convex regression problem and a deep neural network based classification problem are presented to illustrate the effectiveness of the proposed algorithms.

RandPiper Reconfiguration-Friendly Random Beacons with Quadratic Communication

Abstract: A random beacon provides a continuous public source of randomness and its applications range from public lotteries to zero-knowledge proofs. Existing random beacon protocols sacrifice either the fault tolerance or the communication complexity for security, or ease of reconfigurability. This work overcomes the challenges with the existing works through a novel communication efficient combination of state machine replication and (Publicly) Verifiable Secret Sharing (PVSS/VSS). For a system with n nodes in the synchronous communication model and a security parameter κ, we first design an optimally resilient Byzantine fault-tolerant state machine replication protocol with O(κ n2) bits communication per consensus decision without using threshold signatures. Next, we design GRandPiper (Good Pipelined Random beacon), a random beacon protocol with bias-resistance and unpredictability, that uses PVSS and has a communication complexity of O(K n2) always, for a static adversary. However, GRandPiper allows an adaptive adversary to predict beacon values up to t+1 epochs into the future. Therefore, we design BRandPiper (Better RandPiper), that uses VSS and has a communication complexity of O(κ fn2), where f is the actual number of faults, while offering a strong unpredictability with an advantage of only a single round even for an adaptive adversary. We also provide reconfiguration mechanisms to restore the resilience of the beacon protocols while still maintaining quadratic communication complexity per epoch. We implement BRandPiper and compare it against the state-of-the-art practically deployed beacon protocol, Drand, and show that we are always better than or equal to it in performance.

Distributed Second Order Methods with Fast Rates and Compressed Communication

Abstract: We develop several new communication-efficient second-order methods for distributed optimization. Our first method, NEWTON-STAR, is a variant of Newton's method from which it inherits its fast local quadratic rate. However, unlike Newton's method, NEWTON-STAR enjoys the same per iteration communication cost as gradient descent. While this method is impractical as it relies on the use of certain unknown parameters characterizing the Hessian of the objective function at the optimum, it serves as the starting point which enables us design practical variants thereof with strong theoretical guarantees. In particular, we design a stochastic sparsification strategy for learning the unknown parameters in an iterative fashion in a communication efficient manner. Applying this strategy to NEWTON-STAR leads to our next method, NEWTON-LEARN, for which we prove local linear and superlinear rates independent of the condition number. When applicable, this method can have dramatically superior convergence behavior when compared to state-of-the-art methods. Finally, we develop a globalization strategy using cubic regularization which leads to our next method, CUBIC-NEWTON-LEARN, for which we prove global sublinear and linear convergence rates, and a fast superlinear rate. Our results are supported with experimental results on real datasets, and show several orders of magnitude improvement on baseline and state-of-the-art methods in terms of communication complexity.

Efficient Linear Multiparty PSI and Extensions to Circuit/Quorum PSI

Abstract: Multiparty Private Set Intersection (mPSI), enables n parties, each holding private sets (each of size m) to securely compute the intersection of these private sets. While several protocols are known for this task, the only concretely efficient protocol is due to the work of Kolesnikov et al. (KMPRT, CCS 2017), who gave a semi-honest secure protocol with communication complexity O(nmtƛ), where t –First, for the natural adversarial setting of semi-honest honest majority (i.e. t –Second, concretely, our protocol has 6(t+2)/5 times lesser communication than KMPRT and is up to 5× and 6.2× faster than KMPRT in the LAN and WAN setting even for 15 parties. –Finally, we introduce and consider two important variants of mPSI - circuit PSI (that allows the parties to compute a function over the intersection set without revealing the intersection itself) and quorum PSI (that allows P1 to learn all the elements in his/her set that are present in at least k other sets) and provide concretely efficient protocols for these variants.

Hierarchical Federated Learning for Hybrid Data Partitioning Across Multitype Sensors

Abstract: Emerging hardware technology enables the utilization of a large number of multitype sensors with diverse sensing capabilities for data collection and the training of AI models. Each sensor collects partial data samples on a type-specific feature space, which results in hybrid data partitioning across the local datasets and brings challenges to developing a novel communication-efficient and scalable training algorithm. We propose a hierarchical federated learning framework for such hybrid data partitioning with a multitier-partitioned neural network architecture. Specifically, we adopt a primal-dual transform to decompose the training problem on both the sample and feature space. Then, a stochastic coordinate gradient descent ascent algorithm is implemented with intratype and intertype over-the-air aggregation for the update of the primal variables and dual variables, respectively. The incorporation of over-the-air aggregation for signal transmission naturally harnesses the channel perturbations and interference for lower communication complexity and preserved privacy. Despite the influence of transmission noise and channel distortion, convergence analysis is provided for general objective functions, which illustrates the robust training performance of the proposed algorithm with a theoretical guarantee.

Query-to-Communication Lifting Using Low-Discrepancy Gadgets

Abstract: Lifting theorems are theorems that relate the query complexity of a function $f:\{0,1\}^{n}\to \{0,1\}$ to the communication complexity of the composed function $f\circ g^{n}$ for some “gadget” $g:...

Unconditional Communication-Efficient MPC via Hall’s Marriage Theorem

Abstract: The best known n party unconditional multiparty computation protocols with an optimal corruption threshold communicates O(n) field elements per gate. This has been the case even in the semi-honest setting despite over a decade of research on communication complexity in this setting. Going to the slightly sub-optimal corruption setting, the work of Damgard, Ishai, and Kroigaard (EUROCRYPT 2010) provided the first protocol for a single circuit achieving communication complexity of \(O(\log |C|)\) elements per gate. While a number of works have improved upon this result, obtaining a protocol with O(1) field elements per gate has been an open problem.

An efficient missing tag identification approach in RFID collisions

Abstract: Radio frequency identification technology has been widely used to verify the presence of items in many applications such as warehouse management and supply chain logistics. In these applications, the challenge of how to timely identify the missing tags (namely tag searching or missing tag identification) is a key focus. Existing missing tag identification solutions have not achieved their full potentials because collision slots have not been well explored. In this paper, we propose an approach named collision resolving based missing tag identification (CR-MTI) to break through the performance bottleneck of existing missing tag identification protocols. In CR-MTI, multiple tags are allowed to respond with different binary strings in a collision slot. Then, the reader can verify them together by using the bit tracking technology and particularly designed string, thereby significantly improve the time efficiency. CR-MTI also reduces the number of messages transmitted by the reader using customized coding. We further explore the optimal parameter settings to maximize the performance of our proposed CR-MTI. Extensive simulation results show that our proposed CR-MTI outperforms prior art in terms of time efficiency, total executive time and communication complexity.

Faster unbalanced Private Set Intersection in the semi-honest setting

Abstract: Protocols for Private Set Intersection (PSI) are important cryptographic techniques to perform joint operations on datasets in a privacy-preserving way. They allow two parties to compute the intersection of their private sets without revealing any additional information beyond the intersection itself, for one party (one-way) or both parties (mutual). Despite the several PSI protocols available in the literature, only recently techniques have been applied to existing PSI protocols in order to make them more efficient when one of the parties holds a set much smaller than the other. This is a realistic scenario in many cases, characterizing the unbalanced setting. Thus, this paper builds on modern cryptographic engineering techniques and proposes optimizations for a promising one-way PSI protocol based on public-key cryptography secure against semi-honest adversaries. We show that our improvements and optimizations yield a protocol that outperforms the communication complexity and the run time of previous proposals in the unbalanced setting.

A framework for dynamic matching in weighted graphs

Abstract: We introduce a new framework for computing approximate maximum weight matchings. Our primary focus is on the fully dynamic setting, where there is a large gap between the guarantees of the best known algorithms for computing weighted and unweighted matchings. Indeed, almost all current weighted matching algorithms that reduce to the unweighted problem lose a factor of two in the approximation ratio. In contrast, in other sublinear models such as the distributed and streaming models, recent work has largely closed this weighted/unweighted gap. For bipartite graphs, we almost completely settle the gap with a general reduction that converts any algorithm for α-approximate unweighted matching to an algorithm for (1−)α-approximate weighted matching, while only increasing the update time by an O(logn) factor for constant . We also show that our framework leads to significant improvements for non-bipartite graphs, though not in the form of a universal reduction. In particular, we give two algorithms for weighted non-bipartite matching: 1. A randomized (Las Vegas) fully dynamic algorithm that maintains a (1/2−)-approximate maximum weight matching in worst-case update time O(polylog n) with high probability against an adaptive adversary. Our bounds are essentially the same as those of the unweighted algorithm of Wajc [STOC 2020]. 2. A deterministic fully dynamic algorithm that maintains a (2/3−)-approximate maximum weight matching in amortized update time O(m1/4). Our bounds are essentially the same as those of the unweighted algorithm of Bernstein and Stein [SODA 2016]. A key feature of our framework is that it uses existing algorithms for unweighted matching as black-boxes. As a result, our framework is simple and versatile. Moreover, our framework easily translates to other models, and we use it to derive new results for the weighted matching problem in streaming and communication complexity models.

Constant-Overhead Unconditionally Secure Multiparty Computation Over Binary Fields

Abstract: We study the communication complexity of unconditionally secure multiparty computation (MPC) protocols in the honest majority setting. Despite tremendous efforts in achieving efficient protocols for binary fields under computational assumptions, there are no efficient unconditional MPC protocols in this setting. In particular, there are no n-party protocols with constant overhead admitting communication complexity of O(n) bits per gate. Cascudo, Cramer, Xing and Yuan (CRYPTO 2018) were the first ones to achieve such an overhead in the amortized setting by evaluating \(O(\log n)\) copies of the same circuit in the binary field in parallel. In this work, we construct the first unconditional MPC protocol secure against a malicious adversary in the honest majority setting evaluating just a single boolean circuit with amortized communication complexity of O(n) bits per gate.

Data Encoding for Byzantine-Resilient Distributed Optimization

Abstract: We study distributed optimization in the presence of Byzantine adversaries, where both data and computation are distributed among $m$ worker machines, $t$ of which may be corrupt. The compromised nodes may collaboratively and arbitrarily deviate from their pre-specified programs, and a designated (master) node iteratively computes the model/parameter vector for generalized linear models . In this work, we primarily focus on two iterative algorithms: Proximal Gradient Descent (PGD) and Coordinate Descent (CD). Gradient descent (GD) is a special case of these algorithms. PGD is typically used in the data-parallel setting, where data is partitioned across different samples, whereas, CD is used in the model-parallelism setting, where data is partitioned across the parameter space. At the core of our solutions to both these algorithms is a method for Byzantine-resilient matrix-vector (MV) multiplication; and for that, we propose a method based on data encoding and error correction over real numbers to combat adversarial attacks. We can tolerate up to $t\leq \lfloor \frac {m-1}{2}\rfloor $ corrupt worker nodes, which is information-theoretically optimal. We give deterministic guarantees, and our method does not assume any probability distribution on the data. We develop a sparse encoding scheme which enables computationally efficient data encoding and decoding. We demonstrate a trade-off between the corruption threshold and the resource requirements (storage, computational, and communication complexity). As an example, for $t\leq \frac {m}{3}$ , our scheme incurs only a constant overhead on these resources, over that required by the plain distributed PGD/CD algorithms which provide no adversarial protection. To the best of our knowledge, ours is the first paper that connects MV multiplication with CD and designs a specific encoding matrix for MV multiplication whose structure we can leverage to make CD secure against adversarial attacks. Our encoding scheme extends efficiently to (i) the data streaming model, in which data samples come in an online fashion and are encoded as they arrive, and (ii) making stochastic gradient descent (SGD) Byzantine-resilient. In the end, we give experimental results to show the efficacy of our proposed schemes.

Discrete collective estimation in swarm robotics with distributed Bayesian belief sharing

Abstract: Multi-option collective decision-making is a challenging task in the context of swarm intelligence. In this paper, we extend the problem of collective perception from simple binary decision-making of choosing the color in majority to estimating the most likely fill ratio from a series of discrete fill ratio hypotheses. We have applied direct comparison (DC) and direct modulation of voter-based decisions (DMVD) to this scenario to observe their performances in a discrete collective estimation problem. We have also compared their performances against an Individual Exploration baseline. Additionally, we propose a novel collective decision-making strategy called distributed Bayesian belief sharing (DBBS) and apply it to the above discrete collective estimation problem. In the experiments, we explore the performances of considered collective decision-making algorithms in various parameter settings to determine the trade-off among accuracy, speed, message transfer and reliability in the decision-making process. Our results show that both DC and DMVD outperform the Individual Exploration baseline, but both algorithms exhibit different trade-offs with respect to accuracy and decision speed. On the other hand, DBBS exceeds the performances of all other considered algorithms in all four metrics, at the cost of higher communication complexity.

MARINA: Faster Non-Convex Distributed Learning with Compression

Abstract: We develop and analyze MARINA: a new communication efficient method for non-convex distributed learning over heterogeneous datasets. MARINA employs a novel communication compression strategy based on the compression of gradient differences that is reminiscent of but different from the strategy employed in the DIANA method of Mishchenko et al. (2019). Unlike virtually all competing distributed first-order methods, including DIANA, ours is based on a carefully designed biased gradient estimator, which is the key to its superior theoretical and practical performance. The communication complexity bounds we prove for MARINA are evidently better than those of all previous first-order methods. Further, we develop and analyze two variants of MARINA: VR-MARINA and PP-MARINA. The first method is designed for the case when the local loss functions owned by clients are either of a finite sum or of an expectation form, and the second method allows for a partial participation of clients -- a feature important in federated learning. All our methods are superior to previous state-of-the-art methods in terms of oracle/communication complexity. Finally, we provide a convergence analysis of all methods for problems satisfying the Polyak-Lojasiewicz condition.

FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization.

Abstract: We develop two new algorithms, called, FedDR and asyncFedDR, for solving a fundamental nonconvex composite optimization problem in federated learning. Our algorithms rely on a novel combination between a nonconvex Douglas-Rachford splitting method, randomized block-coordinate strategies, and asynchronous implementation. They can also handle convex regularizers. Unlike recent methods in the literature, e.g., FedSplit and FedPD, our algorithms update only a subset of users at each communication round, and possibly in an asynchronous manner, making them more practical. These new algorithms also achieve communication efficiency and more importantly can handle statistical and system heterogeneity, which are the two main challenges in federated learning. Our convergence analysis shows that the new algorithms match the communication complexity lower bound up to a constant factor under standard assumptions. Our numerical experiments illustrate the advantages of our methods compared to existing ones on several datasets.

Distributed Multi-User Secret Sharing

Abstract: We consider a distributed secret sharing system that consists of a dealer, n storage nodes, and m users. Each user is given access to a certain subset of storage nodes, where it can download the stored data. The dealer wants to securely convey a specific secret $\text {s}_{\text {j}}$ to user j via storage nodes, for $\text {j}=1,2, {\dots },\text {m}$ . More specifically, two secrecy conditions are considered in this multi-user context. The weak secrecy condition is that each user does not get any information about the individual secrets of other users, while the perfect secrecy condition implies that a user does not get any information about the collection of all other users’ secrets. In this system, the dealer encodes secrets into several secret shares and loads them into the storage nodes. Given a certain number of storage nodes we find the maximum number of users that can be served in such a system and construct schemes that achieve this with perfect secrecy. We further define two major properties for such distributed secret sharing systems; communication complexity is defined as the total amount of data that users need to download in order to reconstruct their secrets; and storage overhead is defined as the total size of data loaded by the dealer into the storage nodes normalized by the total size of secrets. Lower bounds on the minimum communication complexity and the storage overhead are characterized given any n and m. We construct distributed secret sharing protocols, under certain conditions on the system parameters, that attain the lower bound on the communication complexity while providing perfect secrecy. Furthermore, we construct protocols, again under certain conditions, that simultaneously attain the lower bounds on the communication complexity and the storage overhead while providing weak secrecy, thereby demonstrating schemes that are optimal in terms of both parameters. It is shown how to modify the proposed protocols in order to construct schemes with balanced storage load and communication complexity.

Optimal extension protocols for byzantine broadcast and agreement

Abstract: The problems of Byzantine Broadcast (BB) and Byzantine Agreement (BA) are of interest to both the distributed computing and cryptography communities. Extension protocols for these primitives have been introduced to handle long messages efficiently at the cost of small number of single-bit broadcasts, referred to as seed broadcasts. While the communication optimality has remained the most sought-after property of an extension protocol in the literature, we prioritize both communication and round optimality in this work. In a setting with n parties and a static adversary controlling at most t parties in Byzantine fashion, we present BB and BA extension protocols with $$t