计量经济分析 = Econometric analysis

Gossip algorithms are attractive for in-network processing in sensor networks because they do not require any specialized routing, there is no bottleneck or single point of failure, and they are robust to unreliable wireless network conditions. Recently, there has been a surge of activity in the computer science, control, signal processing, and information theory communities, developing faster and more robust gossip algorithms and deriving theoretical performance guarantees. This paper presents an overview of recent work in the area. We describe convergence rate results, which are related to the number of transmitted messages and thus the amount of energy consumed in the network for gossiping. We discuss issues related to gossiping over wireless links, including the effects of quantization and noise, and we illustrate the use of gossip algorithms for canonical signal processing tasks including distributed estimation, source localization, and compression.

https://users.ece.utexas.edu/~dimakis/Gossip_Survey.pdf

Gossip Algorithms for Distributed Signal Processing

We propose an adaptive diffusion mechanism to optimize global cost functions in a distributed manner over a network of nodes. The cost function is assumed to consist of a collection of individual components. Diffusion adaptation allows the nodes to cooperate and diffuse information in real-time; it also helps alleviate the effects of stochastic gradient noise and measurement noise through a continuous learning process. We analyze the mean-square-error performance of the algorithm in some detail, including its transient and steady-state behavior. We also apply the diffusion algorithm to two problems: distributed estimation with sparse parameters and distributed localization. Compared to well-studied incremental methods, diffusion methods do not require the use of a cyclic path over the nodes and are robust to node and link failure. Diffusion methods also endow networks with adaptation abilities that enable the individual nodes to continue learning even when the cost function changes with time. Examples involving such dynamic cost functions with moving targets are common in the context of biological networks.

Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks

This work deals with the topic of information processing over graphs. The presentation is largely self-contained and covers results that relate to the analysis and design of multi-agent networks for the distributed solution of optimization, adaptation, and learning problems from streaming data through localized interactions among agents. The results derived in this work are useful in comparing network topologies against each other, and in comparing networked solutions against centralized or batch implementations. There are many good reasons for the peaked interest in distributed implementations, especially in this day and age when the word "network" has become commonplace whether one is referring to social networks, power networks, transportation networks, biological networks, or other types of networks. Some of these reasons have to do with the benefits of cooperation in terms of improved performance and improved resilience to failure. Other reasons deal with privacy and secrecy considerations where agents may not be comfortable sharing their data with remote fusion centers. In other situations, the data may already be available in dispersed locations, as happens with cloud computing. One may also be interested in learning through data mining from big data sets. Motivated by these considerations, this work examines the limits of performance of distributed solutions and discusses procedures that help bring forth their potential more fully. The presentation adopts a useful statistical framework and derives performance results that elucidate the mean-square stability, convergence, and steady-state behavior of the learning networks. At the same time, the work illustrates how distributed processing over graphs gives rise to some revealing phenomena due to the coupling effect among the agents. These phenomena are discussed in the context of adaptive networks, along with examples from a variety of areas including distributed sensing, intrusion detection, distributed estimation, online adaptation, network system theory, and machine learning.

/pdf/adaptation-learning-and-optimization-over-networks-h3bow27vho.pdf

Adaptation, Learning, and Optimization Over Networks

This paper surveys recent advances related to adaptation, learning, and optimization over networks. Various distributed strategies are discussed that enable a collection of networked agents to interact locally in response to streaming data and to continually learn and adapt to track drifts in the data and models. Under reasonable technical conditions on the data, the adaptive networks are shown to be mean square stable in the slow adaptation regime, and their mean square error performance and convergence rate are characterized in terms of the network topology and data statistical moments. Classical results for single-agent adaptation and learning are recovered as special cases. The performance results presented in this work are useful in comparing network topologies against each other, and in comparing adaptive networks against centralized or batch implementations. The presentation is complemented with various examples linking together results from various domains.

Adaptive Networks

Motivated by applications to wireless sensor, peer-to-peer, and ad hoc networks, we study distributed broadcasting algorithms for exchanging information and computing in an arbitrarily connected network of nodes. Specifically, we study a broadcasting-based gossiping algorithm to compute the (possibly weighted) average of the initial measurements of the nodes at every node in the network. We show that the broadcast gossip algorithm converges almost surely to a consensus. We prove that the random consensus value is, in expectation, the average of initial node measurements and that it can be made arbitrarily close to this value in mean squared error sense, under a balanced connectivity model and by trading off convergence speed with accuracy of the computation. We provide theoretical and numerical results on the mean square error performance, on the convergence rate and study the effect of the ldquomixing parameterrdquo on the convergence rate of the broadcast gossip algorithm. The results indicate that the mean squared error strictly decreases through iterations until the consensus is achieved. Finally, we assess and compare the communication cost of the broadcast gossip algorithm to achieve a given distance to consensus through theoretical and numerical results.

/pdf/broadcast-gossip-algorithms-for-consensus-uv64rz4x01.pdf

Broadcast Gossip Algorithms for Consensus

In this paper, we develop algorithms for distributed computation of averages of the node data over networks with bandwidth/power constraints or large volumes of data. Distributed averaging algorithms fail to achieve consensus when deterministic uniform quantization is adopted. We propose a distributed algorithm in which the nodes utilize probabilistically quantized information, i.e., dithered quantization, to communicate with each other. The algorithm we develop is a dynamical system that generates sequences achieving a consensus at one of the quantization values almost surely. In addition, we show that the expected value of the consensus is equal to the average of the original sensor data. We derive an upper bound on the mean-square-error performance of the probabilistically quantized distributed averaging (PQDA). Moreover, we show that the convergence of the PQDA is monotonic by studying the evolution of the minimum-length interval containing the node values. We reveal that the length of this interval is a monotonically nonincreasing function with limit zero. We also demonstrate that all the node values, in the worst case, converge to the final two quantization bins at the same rate as standard unquantized consensus. Finally, we report the results of simulations conducted to evaluate the behavior and the effectiveness of the proposed algorithm in various scenarios.

/pdf/distributed-average-consensus-with-dithered-quantization-4yucqluxem.pdf

Distributed Average Consensus With Dithered Quantization

Recent results in compressed sensing show that a sparse or compressible signal can be reconstructed from a few incoherent measurements. Since noise is always present in practical data acquisition systems, sensing, and reconstruction methods are developed assuming a Gaussian (light-tailed) model for the corrupting noise. However, when the underlying signal and/or the measurements are corrupted by impulsive noise, commonly employed linear sampling operators, coupled with current reconstruction algorithms, fail to recover a close approximation of the signal. In this paper, we propose robust methods for sampling and reconstructing sparse signals in the presence of impulsive noise. To solve the problem of impulsive noise embedded in the underlying signal prior the measurement process, we propose a robust nonlinear measurement operator based on the weighed myriad estimator. In addition, we introduce a geometric optimization problem based on L 1 minimization employing a Lorentzian norm constraint on the residual error to recover sparse signals from noisy measurements. Analysis of the proposed methods show that in impulsive environments when the noise posses infinite variance we have a finite reconstruction error and furthermore these methods yield successful reconstruction of the desired signal. Simulations demonstrate that the proposed methods significantly outperform commonly employed compressed sensing sampling and reconstruction techniques in impulsive environments, while providing comparable performance in less demanding, light-tailed environments.

Robust Sampling and Reconstruction Methods for Sparse Signals in the Presence of Impulsive Noise

Speckle is a multiplicative noise that degrades ultrasound images. Recent advancements in ultrasound instrumentation and portable ultrasound devices necessitate the need for more robust despeckling techniques, for both routine clinical practice and teleconsultation. Methods previously proposed for speckle reduction suffer from two major limitations: 1) noise attenuation is not sufficient, especially in the smooth and background areas; 2) existing methods do not sufficiently preserve or enhance edges-they only inhibit smoothing near edges. In this paper, we propose a novel technique that is capable of reducing the speckle more effectively than previous methods and jointly enhancing the edge information, rather than just inhibiting smoothing. The proposed method utilizes the Rayleigh distribution to model the speckle and adopts the robust maximum-likelihood estimation approach. The resulting estimator is statistically analyzed through first and second moment derivations. A tuning parameter that naturally evolves in the estimation equation is analyzed, and an adaptive method utilizing the instantaneous coefficient of variation is proposed to adjust this parameter. To further tailor performance, a weighted version of the proposed estimator is introduced to exploit varying statistics of input samples. Finally, the proposed method is evaluated and compared to well-accepted methods through simulations utilizing synthetic and real ultrasound data

Rayleigh-Maximum-Likelihood Filtering for Speckle Reduction of Ultrasound Images

Decentralized estimation of a noise-corrupted source parameter by a bandwidth-constrained sensor network feeding, through a noisy channel, a fusion center is considered. The sensors, due to bandwidth constraints, provide binary representatives of a noise-corrupted source parameter. Recently, proposed decentralized, distributed estimation, and power scheduling methods do no consider errors occurring during the transmission of binary observations from the sensors to fusion center. In this paper, we extend the decentralized estimation model to the case where imperfect transmission channels are considered. The proposed estimator, which operates on additive channel noise corrupted versions of quantized noisy sensor observations, is approached from maximum likelihood (ML) perspective. The resulting ML estimate is a root, in the region of interest (ROI), of a derivative polynomial function. We analyze the natural logarithm of the polynomial within the ROI showing that the function is log-concave, thereby indicating that numerical methods, such as Newton's algorithm, can be utilized to obtain the optimal solution. Due to complexity and implementation issues associated with the numerical methods, we derive and analyze simpler suboptimal solutions, i.e., the two-stage and mean estimators. The two-stage estimator first estimates the binary observations from noisy fusion center observations utilizing a threshold operation, followed by an estimate of the source parameter. The optimal threshold is the maximum a posteriori (MAP) detector for binary detection and minimizes the probability of binary observation estimation error. Optimal threshold expressions for commonly utilized light-(Gaussian) and heavy-tailed (Cauchy) channel noise models are derived. The mean estimator simply averages the noisy fusion center observations. The output variances of means of the proposed suboptimal estimators are derived. In addition, a computational complexity analysis is presented comparing the proposed ML optimal and suboptimal two-stage and mean estimators. Numerical examples evaluating and comparing the performance of proposed ML, two-stage and mean estimators are also presented.

T.C. Aysal

Papers

Broadcast Gossip Algorithms for Consensus

Distributed Average Consensus With Dithered Quantization

Robust Sampling and Reconstruction Methods for Sparse Signals in the Presence of Impulsive Noise

Rayleigh-Maximum-Likelihood Filtering for Speckle Reduction of Ultrasound Images

Constrained Decentralized Estimation Over Noisy Channels for Sensor Networks