Showing papers on "Distributed algorithm published in 2014"

Fast Distributed Gradient Methods

Abstract: We study distributed optimization problems when N nodes minimize the sum of their individual costs subject to a common vector variable. The costs are convex, have Lipschitz continuous gradient (with constant L), and bounded gradient. We propose two fast distributed gradient algorithms based on the centralized Nesterov gradient algorithm and establish their convergence rates in terms of the per-node communications K and the per-node gradient evaluations k. Our first method, Distributed Nesterov Gradient, achieves rates O( logK/K) and O(logk/k). Our second method, Distributed Nesterov gradient with Consensus iterations, assumes at all nodes knowledge of L and μ(W) - the second largest singular value of the N ×N doubly stochastic weight matrix W. It achieves rates O( 1/ K2-ξ) and O( 1/k2) ( ξ > 0 arbitrarily small). Further, we give for both methods explicit dependence of the convergence constants on N and W. Simulation examples illustrate our findings.

Distributed Optimal Power Flow Using ADMM

Abstract: Distributed optimal power flow (OPF) is a challenging non-linear, non-convex problem of central importance to the future power grid. Although many approaches are currently available in the literature, these require some form of central coordination to properly work. In this paper a fully distributed and robust algorithm for OPF is proposed which does not require any form of central coordination. The algorithm is based upon the alternating direction multiplier method (ADMM) in a form recently proposed by the author, which, in turn, builds upon the work of Schizas The approach is customized as a region-based optimization procedure, and it is tested in meaningful scenarios.

Distributed Consensus-Based Economic Dispatch With Transmission Losses

Abstract: A distributed algorithm is presented to solve the economic power dispatch with transmission line losses and generator constraints. The proposed approach is based on two consensus algorithms running in parallel. The first algorithm is a first-order consensus protocol modified by a correction term which uses a local estimation of the system power mismatch to ensure the generation-demand equality. The second algorithm performs the estimation of the power mismatch in the system using a consensus strategy called consensus on the most up-to-date information. The proposed approach can handle networks of different size and topology using the information about the number of nodes which is also evaluated in a distributed fashion. Simulations performed on standard test cases demonstrate the effectiveness of the proposed approach for both small and large systems.

Distributed Constrained Optimization by Consensus-Based Primal-Dual Perturbation Method

Abstract: Various distributed optimization methods have been developed for solving problems which have simple local constraint sets and whose objective function is the sum of local cost functions of distributed agents in a network. Motivated by emerging applications in smart grid and distributed sparse regression, this paper studies distributed optimization methods for solving general problems which have a coupled global cost function and have inequality constraints. We consider a network scenario where each agent has no global knowledge and can access only its local mapping and constraint functions. To solve this problem in a distributed manner, we propose a consensus-based distributed primal-dual perturbation (PDP) algorithm. In the algorithm, agents employ the average consensus technique to estimate the global cost and constraint functions via exchanging messages with neighbors, and meanwhile use a local primal-dual perturbed subgradient method to approach a global optimum. The proposed PDP method not only can handle smooth inequality constraints but also non-smooth constraints such as some sparsity promoting constraints arising in sparse optimization. We prove that the proposed PDP algorithm converges to an optimal primal-dual solution of the original problem, under standard problem and network assumptions. Numerical results illustrating the performance of the proposed algorithm for a distributed demand response control problem in smart grid are also presented.

Multitask Diffusion Adaptation Over Networks

Abstract: Adaptive networks are suitable for decentralized inference tasks. Recent works have intensively studied distributed optimization problems in the case where the nodes have to estimate a single optimum parameter vector collaboratively. However, there are many important applications that are multitask-oriented in the sense that there are multiple optimum parameter vectors to be inferred simultaneously, in a collaborative manner, over the area covered by the network. In this paper, we employ diffusion strategies to develop distributed algorithms that address multitask problems by minimizing an appropriate mean-square error criterion with l2-regularization. The stability and performance of the algorithm in the mean and mean-square error sense are analyzed. Simulations are conducted to verify the theoretical findings, and to illustrate how the distributed strategy can be used in several useful applications related to target localization and hyperspectral data unmixing.

Incremental Welfare Consensus Algorithm for Cooperative Distributed Generation/Demand Response in Smart Grid

Abstract: In this paper, we introduce the incremental welfare consensus algorithm for solving the energy management problem in a smart grid environment populated with distributed generators and responsive demands. The proposed algorithm is distributed and cooperative such that it eliminates the need for a central energy-management unit, central price coordinator, or leader. The optimum energy solution is found through local peer-to-peer communications among smart devices. Each distributed generation unit is connected to a local price regulator, as is each consumer unit. In response to the price of energy proposed by the local price regulators, the power regulator on each generation/consumer unit determines the level of generation/consumption power needed to optimize the benefit of the device. The consensus-based coordination among price regulators drives the behavior of the overall system toward the global optimum, despite the greedy behavior of each unit. The primary advantages of the proposed approach are: 1) convergence to the global optimum without requiring a central controller/coordinator or leader, despite the greedy behavior at the individual level and limited communications; and 2) scalability in terms of per-node computation and communications burden.

Optimal Distributed Control of Reactive Power Via the Alternating Direction Method of Multipliers

Abstract: We formulate the control of reactive power generation by photovoltaic inverters in a power distribution circuit as a constrained optimization that aims to minimize power losses subject to finite inverter capacity and upper and lower voltage limits at all nodes in the circuit. When voltage variations along the circuit are small and losses of both real and reactive powers are small compared with the respective flows, the resulting optimization problem is convex. Moreover, the cost function is separable enabling a distributed online implementation with node-local computations using only local measurements augmented with limited information from the neighboring nodes communicated over cyber channels. Such an approach lies between the fully centralized and local policy approaches previously considered. We explore protocols based on the dual-ascent method and on the alternating direction method of multipliers (ADMMs), and find that the ADMM protocol performs significantly better.

Joint Mobile Data Gathering and Energy Provisioning in Wireless Rechargeable Sensor Networks

Abstract: The emerging wireless energy transfer technology enables charging sensor batteries in a wireless sensor network (WSN) and maintaining perpetual operation of the network. Recent breakthrough in this area has opened up a new dimension to the design of sensor network protocols. In the meanwhile, mobile data gathering has been considered as an efficient alternative to data relaying in WSNs. However, time variation of recharging rates in wireless rechargeable sensor networks imposes a great challenge in obtaining an optimal data gathering strategy. In this paper, we propose a framework of joint wireless energy replenishment and anchor-point based mobile data gathering (WerMDG) in WSNs by considering various sources of energy consumption and time-varying nature of energy replenishment. To that end, we first determine the anchor point selection strategy and the sequence to visit the anchor points. We then formulate the WerMDG problem into a network utility maximization problem which is constrained by flow, energy balance, link and battery capacity and the bounded sojourn time of the mobile collector. Furthermore, we present a distributed algorithm composed of cross-layer data control, scheduling and routing subalgorithms for each sensor node, and sojourn time allocation subalgorithm for the mobile collector at different anchor points. We also provide the convergence analysis of these subalgorithms. Finally, we implement the WerMDG algorithm in a distributed manner in the NS-2 simulator and give extensive numerical results to verify the convergence of the proposed algorithm and the impact of utility weight, link capacity and recharging rate on network performance.

A Distributed Algorithm of Appliance Scheduling for Home Energy Management System

Abstract: Demand side management encourages the users in a smart grid to shift their electricity consumption in response to varying electricity prices. In this paper, we propose a distributed framework for the demand response based on cost minimization. Each user in the system will find an optimal start time and operating mode for the appliances in response to the varying electricity prices. We model the cost function for each user and the constraints for the appliances. We then propose an approximate greedy iterative algorithm that can be employed by each user to schedule appliances. In the proposed algorithm, each user requires only the knowledge of the price of the electricity, which depends on the aggregated load of other users, instead of the load profiles of individual users. In order for the users to coordinate with each other, we introduce a penalty term in the cost function, which penalizes large changes in the scheduling between successive iterations. Numerical simulations show that our optimization method will result in lower cost for the consumers, lower generation costs for the utility companies, lower peak load, and lower load fluctuations.

An Optimal and Distributed Demand Response Strategy With Electric Vehicles in the Smart Grid

Abstract: In this paper, we propose a new model of demand response management for the future smart grid that integrates plug-in electric vehicles and renewable distributed generators. A price scheme considering fluctuation cost is developed. We consider a market where users have the flexibility to sell back the energy generated from their distributed generators or the energy stored in their plug-in electric vehicles. A distributed optimization algorithm based on the alternating direction method of multipliers is developed to solve the optimization problem, in which consumers need to report their aggregated loads only to the utility company, thus ensuring their privacy. Consumers can update their loads scheduling simultaneously and locally to speed up the optimization computing. Using numerical examples, we show that the demand curve is flattened after the optimization, even though there are uncertainties in the model, thus reducing the cost paid by the utility company. The distributed algorithms are also shown to reduce the users' daily bills.

A Distributed Algorithm for Managing Residential Demand Response in Smart Grids

Abstract: Demand response enabled by time-varying prices can propel the power industry toward a greater efficiency. However, a noncoordinated response of customers may lead to severe peak rebounds at periods with lower prices. In this regard, a coordinated demand response scheme can mitigate concerns about the peak rebounds. This paper presents a system-wide demand response management model to coordinate demand response provided by residential customers. The objective of the model is to flatten the total load profile that is subject to minimum individual cost of customers. The model is first formulated as a bi-level optimization problem. It is then casted into equivalent single-level problems, which are solved via an iterative distributed algorithm. Home load management (HLM) modules embedded in customers' smart meters are autonomous agents associated with the algorithm. In the algorithm, at first, HLM modules, in response to prices announced by the utility, optimize the daily operation of household appliances and send back the scheduled load profiles. Then, the total load profile is calculated and released by the utility. Thereafter, the HLM modules asynchronously update their schedule such that, given their least energy expenses, the most evenly distributed total load profile is achieved. The mutual interaction between the utility and HLM modules is continued to the point in which no further improvement is obtained. Convergence and optimality of the algorithm are proved.

Decentralized Learning for Multiplayer Multiarmed Bandits

Abstract: We consider the problem of distributed online learning with multiple players in multiarmed bandit (MAB) models. Each player can pick among multiple arms. When a player picks an arm, it gets a reward. We consider both independent identically distributed (i.i.d.) reward model and Markovian reward model. In the i.i.d. model, each arm is modeled as an i.i.d. process with an unknown distribution with an unknown mean. In the Markovian model, each arm is modeled as a finite, irreducible, aperiodic and reversible Markov chain with an unknown probability transition matrix and stationary distribution. The arms give different rewards to different players. If two players pick the same arm, there is a collision, and neither of them get any reward. There is no dedicated control channel for coordination or communication among the players. Any other communication between the users is costly and will add to the regret. We propose an online index-based distributed learning policy called dUCB4 algorithm that trades off exploration versus exploitation in the right way, and achieves expected regret that grows at most as near- O(log2T). The motivation comes from opportunistic spectrum access by multiple secondary users in cognitive radio networks wherein they must pick among various wireless channels that look different to different users. This is the first distributed learning algorithm for multiplayer MABs with heterogeneous players (that have player-dependent rewards) to the best of our knowledge.

Distributed State Estimation and Energy Management in Smart Grids: A Consensus ${+}$ Innovations Approach

Abstract: This paper reviews signal processing research for applications in the future electric power grid, commonly referred to as smart grid. Generally, it is expected that the grid of the future would differ from the current system by the increased integration of distributed generation, distributed storage, demand response, power electronics, and communications and sensing technologies. The consequence is that the physical structure of the system becomes significantly more distributed. The existing centralized control structure is not suitable any more to operate such a highly distributed system. Hence, in this paper, we overview distributed approaches, all based on consensus ${+}$ innovations, for three common energy management functions: state estimation, economic dispatch, and optimal power flow. We survey the pertinent literature and summarize our work. Simulation results illustrate tradeoffs and the performance of consensus ${+}$ innovations for these three applications.

Differentially Private Distributed Constrained Optimization

Abstract: Many resource allocation problems can be formulated as an optimization problem whose constraints contain sensitive information about participating users. This paper concerns solving this kind of optimization problem in a distributed manner while protecting the privacy of user information. Without privacy considerations, existing distributed algorithms normally consist in a central entity computing and broadcasting certain public coordination signals to participating users. However, the coordination signals often depend on user information, so that an adversary who has access to the coordination signals can potentially decode information on individual users and put user privacy at risk. We present a distributed optimization algorithm that preserves differential privacy, which is a strong notion that guarantees user privacy regardless of any auxiliary information an adversary may have. The algorithm achieves privacy by perturbing the public signals with additive noise, whose magnitude is determined by the sensitivity of the projection operation onto user-specified constraints. By viewing the differentially private algorithm as an implementation of stochastic gradient descent, we are able to derive a bound for the suboptimality of the algorithm. We illustrate the implementation of our algorithm via a case study of electric vehicle charging. Specifically, we derive the sensitivity and present numerical simulations for the algorithm. Through numerical simulations, we are able to investigate various aspects of the algorithm when being used in practice, including the choice of step size, number of iterations, and the trade-off between privacy level and suboptimality.

Distributed resource allocation for MISO downlink systems via the alternating direction method of multipliers

Abstract: We provide distributed algorithms for the radio resource allocation problem in multicell downlink multi-input single-output systems. Specifically, the problems of (1) minimizing total transmit power subject to signal-to-interference-plus-noise ratio (SINR) constraints of each user and (2) SINR balancing subject to total transmit power constraints are considered. We propose a consensus-based distributed algorithm and the fast solution method via alternating the direction method of multipliers. First, we derive a distributed algorithm for minimization of total transmit power. Then, in conjunction with the bracketing method, a distributed algorithm for SINR balancing is derived. Numerical results show that the proposed distributed algorithms achieve the optimal centralized solution.

Distributed Power Splitting for SWIPT in Relay Interference Channels using Game Theory

Abstract: In this paper, we consider simultaneous wireless information and power transfer (SWIPT) in relay interference channels, where multiple source-destination pairs communicate through their dedicated energy harvesting relays. Each relay needs to split its received signal from sources into two streams: one for information forwarding and the other for energy harvesting. We develop a distributed power splitting framework using game theory to derive a profile of power splitting ratios for all relays that can achieve a good network-wide performance. Specifically, non-cooperative games are respectively formulated for pure amplify-and-forward (AF) and decode-and-forward (DF) networks, in which each link is modeled as a strategic player who aims to maximize its own achievable rate. The existence and uniqueness for the Nash equilibriums (NEs) of the formulated games are analyzed and a distributed algorithm with provable convergence to achieve the NEs is also developed. Subsequently, the developed framework is extended to the more general network setting with mixed AF and DF relays. All the theoretical analyses are validated by extensive numerical results. Simulation results show that the proposed game-theoretical approach can achieve a near-optimal network-wide performance on average, especially for the scenarios with relatively low and moderate interference.

Distributed Maximum Likelihood Sensor Network Localization

Abstract: We propose a class of convex relaxations to solve the sensor network localization problem, based on a maximum likelihood (ML) formulation. This class, as well as the tightness of the relaxations, depends on the noise probability density function (PDF) of the collected measurements. We derive a computational efficient edge-based version of this ML convex relaxation class and we design a distributed algorithm that enables the sensor nodes to solve these edge-based convex programs locally by communicating only with their close neighbors. This algorithm relies on the alternating direction method of multipliers (ADMM), it converges to the centralized solution, it can run asynchronously, and it is computation error-resilient. Finally, we compare our proposed distributed scheme with other available methods, both analytically and numerically, and we argue the added value of ADMM, especially for large-scale networks.

Mobility and Intruder Prior Information Improving the Barrier Coverage of Sparse Sensor Networks

Abstract: The barrier coverage problem in emerging mobile sensor networks has been an interesting research issue due to many related real-life applications. Existing solutions are mainly concerned with deciding one-time movement for individual sensors to construct as many barriers as possible, which may not be suitable when there are no sufficient sensors to form a single barrier. In this paper, we aim to achieve barrier coverage in the sensor scarcity scenario by dynamic sensor patrolling. Specifically, we design a periodic monitoring scheduling (PMS) algorithm in which each point along the barrier line is monitored periodically by mobile sensors. Based on the insight from PMS, we then propose a coordinated sensor patrolling (CSP) algorithm to further improve the barrier coverage, where each sensor's current movement strategy is derived from the information of intruder arrivals in the past. By jointly exploiting sensor mobility and intruder arrival information, CSP is able to significantly enhance barrier coverage. We prove that the total distance that sensors move during each time slot in CSP is the minimum. Considering the decentralized nature of mobile sensor networks, we further introduce two distributed versions of CSP: S-DCSP and G-DCSP. We study the scenario where sensors are moving on two barriers and propose two heuristic algorithms to guide the movement of sensors. Finally, we generalize our results to work for different intruder arrival models. Through extensive simulations, we demonstrate that the proposed algorithms have desired barrier coverage performances.

Real and Complex Monotone Communication Games

Abstract: Noncooperative game-theoretic tools have been increasingly used to study many important resource allocation problems in communications, networking, smart grids, and portfolio optimization. In this paper, we consider a general class of convex Nash equilibrium problems (NEPs), where each player aims at solving an arbitrary smooth convex optimization problem. Differently from most of current works, we do not assume any specific structure for the players' problems, and we allow the optimization variables of the players to be matrices in the complex domain. Our main contribution is the design of a novel class of distributed (asynchronous) best-response-algorithms suitable for solving the proposed NEPs, even in the presence of multiple solutions. The new methods, whose convergence analysis is based on variational inequality (VI) techniques, can select, among all the equilibria of a game, those that optimize a given performance criterion, at the cost of limited signaling among the players. This is a major departure from existing best-response algorithms, whose convergence conditions imply the uniqueness of the NE. Some of our results hinge on the use of VI problems directly in the complex domain; the study of these new kind of VIs also represents a noteworthy innovative contribution. We then apply the developed methods to solve some new generalizations of Single Input Single Output (SISO) and Multiple Input Multiple Output (MIMO) games in cognitive radio systems, showing a considerable performance improvement over classical pure noncooperative schemes.

Cooperative Distributed Demand Management for Community Charging of PHEV/PEVs Based on KKT Conditions and Consensus Networks

Abstract: Efficient and reliable demand side management techniques for community charging of plug-in hybrid electrical vehicles (PHEVs) and plug-in electrical vehicles (PEVs) are needed, as large numbers of these vehicles are being introduced to the power grid. To avoid overloads and maximize customer preferences in terms of time and cost of charging, a constrained nonlinear optimization problem can be formulated. In this paper, we have developed a novel cooperative distributed algorithm for charging control of PHEVs/PEVs that solves the constrained nonlinear optimization problem using Karush-Kuhn-Tucker (KKT) conditions and consensus networks in a distributed fashion. In our design, the global optimal power allocation under all local and global constraints is reached through peer-to-peer coordination of charging stations. Therefore, the need for a central control unit is eliminated. In this way, single-node congestion is avoided when the size of the problem is increased and the system gains robustness against single-link/node failures. Furthermore, via Monte Carlo simulations, we have demonstrated that the proposed distributed method is scalable with the number of charging points and returns solutions, which are comparable to centralized optimization algorithms with a maximum of 2% sub-optimality. Thus, the main advantages of our approach are eliminating the need for a central energy management/coordination unit, gaining robustness against single-link/node failures, and being scalable in terms of single-node computations.

A Distributed Auction-Based Algorithm for the Nonconvex Economic Dispatch Problem

Abstract: This paper presents a distributed algorithm based on auction techniques and consensus protocols to solve the nonconvex economic dispatch problem. The optimization problem of the nonconvex economic dispatch includes several constraints such as valve-point loading effect, multiple fuel option, and prohibited operating zones. Each generating unit locally evaluates quantities used as bids in the auction mechanism. These units send their bids to their neighbors in a communication graph that supports the power system and which provides the required information flow. A consensus procedure is used to share the bids among the network agents and resolves the auction. As a result, the power distribution of generating units is updated and the generation cost is minimized. The effectiveness of this approach is demonstrated by simulations on standard test systems.

SEECH: Scalable Energy Efficient Clustering Hierarchy Protocol in Wireless Sensor Networks

Abstract: The energy efficiency is an important issue for employ distributed wireless sensor networks in smart space and extreme environments. The cluster-based communication protocols play a considerable role for energy saving in hierarchical wireless sensor networks. In most of traditional clustering algorithms, a cluster head (CH) simultaneously serves as a relay sensor node to transmit its cluster/other clusters data packet(s) to the data sink. As a result, each node would have CH role as many as relay role during network lifetime. In our view, this is inefficient from an energy efficiency perspective because in lots of cases, a node due to its position in the network comparatively is more proper to work as a CH and/a relay. This paper proposes a new distributed algorithm named scalable energy efficient clustering hierarchy (SEECH), which selects CHs and relays separately and based on nodes eligibilities. In this way, high and low degree nodes are, respectively, employed as CHs and relays. In only a few past researches, CHs and relays are different, but their goal was mainly mitigation of CHs energy burden which is intrinsically satisfied by the proposed mechanism. To consider uniformity of CHs to balance clusters, SEECH uses a new distance-based algorithm. Comparisons with LEACH and TCAC protocols show obvious better performance of SEECH in term of lifetime. To evaluate the scalability of SEECH strategy, simulations are conducted in three different network size scenarios.

Average Consensus in the Presence of Delays in Directed Graph Topologies

Abstract: Classical distributed algorithms for asymptotic average consensus typically assume timely and reliable exchange of information between neighboring components of a given multi-component system. These assumptions are not necessarily valid in practice due to varying delays that might affect computations at different nodes and/or transmissions at different links. In this work, we propose a protocol that overcomes this limitation and, unlike existing consensus protocols in the presence of delays, ensures asymptotic consensus to the exact average, despite the presence of arbitrary (but bounded) delays in the communication links. The protocol requires that each component has knowledge of the number of its out-neighbors (i.e., the number of components to which it can send information) and its proof of correctness relies on the weak convergence of a backward product of column stochastic matrices. The proposed algorithm is demonstrated via illustrative examples.

A college admissions game for uplink user association in wireless small cell networks

Abstract: In this paper, the problem of uplink user association in small cell networks, which involves interactions between users, small cell base stations, and macro-cell stations, having often conflicting objectives, is considered. The problem is formulated as a college admissions game with transfers in which a number of colleges, i.e., small cell and macro-cell stations seek to recruit a number of students, i.e., users. In this game, the users and access points (small cells and macro-cells) rank one another based on preference functions that capture the users' need to optimize their utilities which are functions of packet success rate (PSR) and delay as well as the small cells' incentive to extend the macro-cell coverage (e.g., via cell biasing/range expansion) while maintaining the users' quality-of-service. A distributed algorithm that combines notions from matching theory and coalitional games is proposed to solve the game. The convergence of the algorithm is shown and the properties of the resulting assignments are discussed. Simulation results show that the proposed approach yields a performance improvement, in terms of the average utility per user, reaching up to 23% relative to a conventional, best-PSR algorithm. I. INTRODUCTION Meeting the stringent quality-of-service (QoS) requirements of emerging wireless services warrants substantial changes in current cellular infrastructure. In this respect, the introduction of small cell base stations (SCBSs) (picocells, microcells, fem-

Distributed approximation algorithms for weighted shortest paths

Abstract: A distributed network is modeled by a graph having n nodes (processors) and diameter D. We study the time complexity of approximating weighted (undirected) shortest paths on distributed networks with a O (log n) bandwidth restriction on edges (the standard synchronous CONGEST model). The question whether approximation algorithms help speed up the shortest paths and distance computation (more precisely distance computation) was raised since at least 2004 by Elkin (SIGACT News 2004). The unweighted case of this problem is well-understood while its weighted counterpart is fundamental problem in the area of distributed approximation algorithms and remains widely open. We present new algorithms for computing both single-source shortest paths (SSSP) and all-pairs shortest paths (APSP) in the weighted case. Our main result is an algorithm for SSSP. Previous results are the classic O(n)-time Bellman-Ford algorithm and an O(n1/2+1/2k + D)-time (8k⌈log(k + 1)⌉ --1)-approximation algorithm, for any integer k ≥ 1, which follows from the result of Lenzen and Patt-Shamir (STOC 2013). (Note that Lenzen and Patt-Shamir in fact solve a harder problem, and we use O(·) to hide the O(poly log n) term.) We present an O (n1/2D1/4 + D)-time (1 + o(1))-approximation algorithm for SSSP. This algorithm is sublinear-time as long as D is sublinear, thus yielding a sublinear-time algorithm with almost optimal solution. When D is small, our running time matches the lower bound of Ω(n1/2 + D) by Das Sarma et al. (SICOMP 2012), which holds even when D=Θ(log n), up to a poly log n factor. As a by-product of our technique, we obtain a simple O (n)-time (1+ o(1))-approximation algorithm for APSP, improving the previous O(n)-time O(1)-approximation algorithm following from the results of Lenzen and Patt-Shamir. We also prove a matching lower bound. Our techniques also yield an O(n1/2) time algorithm on fully-connected networks, which guarantees an exact solution for SSSP and a (2+ o(1))-approximate solution for APSP. All our algorithms rely on two new simple tools: light-weight algorithm for bounded-hop SSSP and shortest-path diameter reduction via shortcuts. These tools might be of an independent interest and useful in designing other distributed algorithms.

NOMAD: non-locking, stochastic multi-machine algorithm for asynchronous and decentralized matrix completion

Abstract: We develop an efficient parallel distributed algorithm for matrix completion, named NOMAD (Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion). NOMAD is a decentralized algorithm with non-blocking communication between processors. One of the key features of NOMAD is that the ownership of a variable is asynchronously transferred between processors in a decentralized fashion. As a consequence it is a lock-free parallel algorithm. In spite of being asynchronous, the variable updates of NOMAD are serializable, that is, there is an equivalent update ordering in a serial implementation. NOMAD outperforms synchronous algorithms which require explicit bulk synchronization after every iteration: our extensive empirical evaluation shows that not only does our algorithm perform well in distributed setting on commodity hardware, but also outperforms state-of-the-art algorithms on a HPC cluster both in multi-core and distributed memory settings.

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Abstract: Gaussian processes (GPs) are a powerful tool for probabilistic inference over functions. They have been applied to both regression and non-linear dimensionality reduction, and offer desirable properties such as uncertainty estimates, robustness to over-fitting, and principled ways for tuning hyper-parameters. However the scalability of these models to big datasets remains an active topic of research. We introduce a novel re-parametrisation of variational inference for sparse GP regression and latent variable models that allows for an efficient distributed algorithm. This is done by exploiting the decoupling of the data given the inducing points to re-formulate the evidence lower bound in a Map-Reduce setting. We show that the inference scales well with data and computational resources, while preserving a balanced distribution of the load among the nodes. We further demonstrate the utility in scaling Gaussian processes to big data. We show that GP performance improves with increasing amounts of data in regression (on flight data with 2 million records) and latent variable modelling (on MNIST). The results show that GPs perform better than many common models often used for big data.

Joint Beamforming and Power Splitting for MISO Interference Channel With SWIPT: An SOCP Relaxation and Decentralized Algorithm

Abstract: This paper considers a power splitting-based MISO interference channel for simultaneous wireless information and power transfer (SWIPT), where each single antenna receiver splits the received signal into two streams of different power for decoding information and harvesting energy separately. We aim to minimize the total transmission power by joint beamforming and power splitting (JBPS) under both the signal-to-interference-plus-noise ratio (SINR) constraints and energy harvesting (EH) constraints. The JBPS problem is nonconvex and has not yet been well addressed in the literature. Moreover, decentralized algorithm design for JBPS based on local channel state information (CSI) and limited information exchange remains open. In this paper, we first propose a novel relaxation method named second-order cone programming (SOCP) relaxation to address the JBPS problem. We formulate the relaxed problem as an SOCP and present two sufficient conditions under which the SOCP relaxation is tight. For the case when the SOCP solution is not necessarily optimal to the JBPS problem, a closed-form feasible-solution-recovery method is provided. Then, we develop a distributed algorithm for the JBPS problem based on primal-decomposition (PD) method. The PD-based distributed algorithm consists of a master problem and a set of subproblems. The former is solved by using subgradient method while the latter are solved using coordinate descent method. Finally, numerical results validates the efficiency of the proposed algorithms.

Distributed 3-D Localization of Camera Sensor Networks From 2-D Image Measurements

Abstract: In this paper we propose distributed algorithms that use 2-D image measurements to estimate the absolute 3-D poses of the nodes in a camera network, with the purpose of enabling higher-level tasks such as tracking and recognition. We assume that pairs of cameras with overlapping fields of view can estimate their relative 3-D pose (rotation and translation direction) using standard computer vision techniques. The solution we propose combines these local, noisy estimates into a single consistent localization. We derive our algorithms from optimization problems on the manifold of poses. We provide theoretical results on the convergence of the algorithms (choice of the step-size, initialization) and on the properties of their solutions (sensitivity, uniqueness). We also provide experiments on synthetic and real data. Interestingly, our algorithm for estimating the rotation part of the poses shows some degree of robustness to outliers.