scispace - formally typeset
Search or ask a question

Showing papers by "Ali H. Sayed published in 2012"


Journal ArticleDOI
TL;DR: An adaptive diffusion mechanism to optimize global cost functions in a distributed manner over a network of nodes, which endow networks with adaptation abilities that enable the individual nodes to continue learning even when the cost function changes with time.
Abstract: We propose an adaptive diffusion mechanism to optimize global cost functions in a distributed manner over a network of nodes. The cost function is assumed to consist of a collection of individual components. Diffusion adaptation allows the nodes to cooperate and diffuse information in real-time; it also helps alleviate the effects of stochastic gradient noise and measurement noise through a continuous learning process. We analyze the mean-square-error performance of the algorithm in some detail, including its transient and steady-state behavior. We also apply the diffusion algorithm to two problems: distributed estimation with sparse parameters and distributed localization. Compared to well-studied incremental methods, diffusion methods do not require the use of a cyclic path over the nodes and are robust to node and link failure. Diffusion methods also endow networks with adaptation abilities that enable the individual nodes to continue learning even when the cost function changes with time. Examples involving such dynamic cost functions with moving targets are common in the context of biological networks.

672 citations


Journal ArticleDOI
TL;DR: It is confirmed that under constant step-sizes, diffusion strategies allow information to diffuse more thoroughly through the network and this property has a favorable effect on the evolution of the network: diffusion networks are shown to converge faster and reach lower mean-square deviation than consensus networks, and their mean- square stability is insensitive to the choice of the combination weights.
Abstract: Adaptive networks consist of a collection of nodes with adaptation and learning abilities. The nodes interact with each other on a local level and diffuse information across the network to solve estimation and inference tasks in a distributed manner. In this work, we compare the mean-square performance of two main strategies for distributed estimation over networks: consensus strategies and diffusion strategies. The analysis in the paper confirms that under constant step-sizes, diffusion strategies allow information to diffuse more thoroughly through the network and this property has a favorable effect on the evolution of the network: diffusion networks are shown to converge faster and reach lower mean-square deviation than consensus networks, and their mean-square stability is insensitive to the choice of the combination weights. In contrast, and surprisingly, it is shown that consensus networks can become unstable even if all the individual nodes are stable and able to solve the estimation task on their own. When this occurs, cooperation over the network leads to a catastrophic failure of the estimation task. This phenomenon does not occur for diffusion networks: we show that stability of the individual nodes always ensures stability of the diffusion network irrespective of the combination topology. Simulation results support the theoretical findings.

414 citations


Journal ArticleDOI
TL;DR: This paper investigates the mean-square performance of general adaptive diffusion algorithms in the presence of various sources of imperfect information exchanges, quantization errors, and model non-stationarities and reveals that link noise over the regression data modifies the dynamics of the network evolution in a distinct way, and leads to biased estimates in steady-state.
Abstract: Adaptive networks rely on in-network and collaborative processing among distributed agents to deliver enhanced performance in estimation and inference tasks. Information is exchanged among the nodes, usually over noisy links. The combination weights that are used by the nodes to fuse information from their neighbors play a critical role in influencing the adaptation and tracking abilities of the network. This paper first investigates the mean-square performance of general adaptive diffusion algorithms in the presence of various sources of imperfect information exchanges, quantization errors, and model non-stationarities. Among other results, the analysis reveals that link noise over the regression data modifies the dynamics of the network evolution in a distinct way, and leads to biased estimates in steady-state. The analysis also reveals how the network mean-square performance is dependent on the combination weights. We use these observations to show how the combination weights can be optimized and adapted. Simulation results illustrate the theoretical findings and match well with theory.

168 citations


Journal ArticleDOI
TL;DR: This work analyzes the mean-square performance of different strategies for distributed estimation over least-mean-squares (LMS) adaptive networks and establishes that, for sufficiently small step-sizes, diffusion strategies can outperform centralized block or incremental LMS strategies by optimizing over left-stochastic combination weighting matrices.
Abstract: In this work, we analyze the mean-square performance of different strategies for distributed estimation over least-mean-squares (LMS) adaptive networks. The results highlight some useful properties for distributed adaptation in comparison to fusion-based centralized solutions. The analysis establishes that, by optimizing over the combination weights, diffusion strategies can deliver lower excess-mean-square-error than centralized solutions employing traditional block or incremental LMS strategies. We first study in some detail the situation involving combinations of two adaptive agents and then extend the results to generic N -node ad-hoc networks. In the latter case, we establish that, for sufficiently small step-sizes, diffusion strategies can outperform centralized block or incremental LMS strategies by optimizing over left-stochastic combination weighting matrices. The results suggest more efficient ways for organizing and processing data at fusion centers, and present useful adaptive strategies that are able to enhance performance when implemented in a distributed manner.

150 citations


Proceedings ArticleDOI
28 May 2012
TL;DR: This work devise an adaptive combination rule that allows agents to learn which neighbors belong to the same cluster and which other neighbors should be ignored, and enables the agents to identify their grouping and to attain improved learning and estimation performance over networks.
Abstract: Distributed processing over networks relies on in-network processing and cooperation among neighboring agents. Cooperation is beneficial when all agents share the same objective or belong to the same group. However, if agents belong to different clusters or are interested in different objectives, then cooperation can be damaging. In this work, we devise an adaptive combination rule that allows agents to learn which neighbors belong to the same cluster and which other neighbors should be ignored. In doing so, the resulting algorithm enables the agents to identify their grouping and to attain improved learning and estimation performance over networks.

85 citations


Proceedings ArticleDOI
01 Nov 2012
TL;DR: It is shown that the diffusion algorithm converges almost surely to the true state and the superior convergence rate of the diffusion strategy over consensus-based strategies since diffusion schemes allow information to diffuse more thoroughly through the network.
Abstract: We propose a diffusion strategy to enable social learning over networks Individual agents observe signals influenced by the state of the environment The individual measurements are not sufficient to enable the agents to detect the true state of the environment on their own Agents are then encouraged to cooperate through a diffusive process of self-learning and social-learning We show that the diffusion algorithm converges almost surely to the true state Simulation results also illustrate the superior convergence rate of the diffusion strategy over consensus-based strategies since diffusion schemes allow information to diffuse more thoroughly through the network

67 citations


Posted Content
TL;DR: In this article, the authors provide an overview of diffusion strategies for adaptation and learning over networks, and compare the performance of cooperative and adaptive diffusion strategies relative to non-cooperative agents.
Abstract: Adaptive networks are well-suited to perform decentralized information processing and optimization tasks and to model various types of self-organized and complex behavior encountered in nature. Adaptive networks consist of a collection of agents with processing and learning abilities. The agents are linked together through a connection topology, and they cooperate with each other through local interactions to solve distributed optimization, estimation, and inference problems in real-time. The continuous diffusion of information across the network enables agents to adapt their performance in relation to streaming data and network conditions; it also results in improved adaptation and learning performance relative to non-cooperative agents. This article provides an overview of diffusion strategies for adaptation and learning over networks. The article is divided into several sections: 1. Motivation; 2. Mean-Square-Error Estimation; 3. Distributed Optimization via Diffusion Strategies; 4. Adaptive Diffusion Strategies; 5. Performance of Steepest-Descent Diffusion Strategies; 6. Performance of Adaptive Diffusion Strategies; 7. Comparing the Performance of Cooperative Strategies; 8. Selecting the Combination Weights; 9. Diffusion with Noisy Information Exchanges; 10. Extensions and Further Considerations; Appendix A: Properties of Kronecker Products; Appendix B: Graph Laplacian and Network Connectivity; Appendix C: Stochastic Matrices; Appendix D: Block Maximum Norm; Appendix E: Comparison with Consensus Strategies; References.

49 citations


Journal ArticleDOI
TL;DR: Diffusion adaptation is used to model the adaptation process in the presence of asymmetric nodes and noisy data and indicates that the models are able to emulate the swarming behavior of bees under varied conditions such as a small number of informed bees, sharing of target location, shares of target direction, and noisy measurements.
Abstract: Honeybees swarm when they move to a new site for their hive. During the process of swarming, their behavior can be analyzed by classifying them as informed bees or uninformed bees, where the informed bees have some information about the destination while the uninformed bees follow the informed bees. The swarm's movement can be viewed as a network of mobile nodes with asymmetric information exchange about their destination. In these networks, adaptive and mobile agents share information on the fly and adapt their estimates in response to local measurements and data shared with neighbors. Diffusion adaptation is used to model the adaptation process in the presence of asymmetric nodes and noisy data. The simulations indicate that the models are able to emulate the swarming behavior of bees under varied conditions such as a small number of informed bees, sharing of target location, sharing of target direction, and noisy measurements.

43 citations


Proceedings ArticleDOI
25 Mar 2012
TL;DR: An adaptive diffusion strategy with limited communication overhead by cutting off all links but one for each node in the network and keeping the “best” neighbor that has the smallest estimated variance-product measure is proposed.
Abstract: We propose an adaptive diffusion strategy with limited communication overhead by cutting off all links but one for each node in the network. We keep the “best” neighbor that has the smallest estimated variance-product measure and ignore the other neighbors. The combination coefficients for the interacting nodes are calculated via a maximal-ratio-combining rule to minimize the steady-state meansquare-deviation. Simulation results illustrate that, with less communication overhead and less computations, the proposed algorithm performs well and outperforms other related methods with similar overheads.

39 citations


Proceedings ArticleDOI
01 Oct 2012
TL;DR: What aspects of the combination policies determine the nature of the Pareto-optimal solution and how close the distributed solution gets to it is revealed and useful constructive procedures to control the convergence behavior of distributed strategies are suggested.
Abstract: Motivated by recent developments in the context of adaptation over networks, this work establishes useful results about the limiting global behavior of diffusion and consensus strategies for the solution of distributed optimization problems. It is known that the choice of combination policies has a direct bearing on the convergence and performance of distribued solutions. This article reveals what aspects of the combination policies determine the nature of the Pareto-optimal solution and how close the distributed solution gets to it. The results suggest useful constructive procedures to control the convergence behavior of distributed strategies and to design effective combination procedures.

39 citations


Proceedings ArticleDOI
25 Mar 2012
TL;DR: Convergence and performance analysis is provided of the proposed diffusion LMS techniques for distributed estimation over adaptive networks, which are able to exploit sparsity in the underlying system model.
Abstract: The goal of this paper is to propose diffusion LMS techniques for distributed estimation over adaptive networks, which are able to exploit sparsity in the underlying system model. The approach relies on convex regularization, common in compressive sensing, to improve the performance of the diffusion strategies. We provide convergence and performance analysis of the proposed method, showing under what conditions it outperforms the unregularized diffusion version. Simulation results illustrate the advantage of the proposed filter under the sparsity assumption on the true coefficient vector.

Journal ArticleDOI
TL;DR: The results indicate that whether temporal processing is performed before or after adaptation, the strategy that performs adaptation before spatial cooperation leads to smaller mean-square error.
Abstract: We present diffusion algorithms for distributed estimation and detection over networks that endow all nodes with both spatial cooperation abilities and temporal processing abilities. Each node in the network is allowed to share information locally with its neighbors; this step amounts to sharing and processing of spatial data. At the same time, each node is allowed to after and process past estimates to improve estimation accuracy through an overall collaborative process. In this manner, the resulting distributed algorithms consist of three stages: adaptation, spatial processing, and temporal processing. Moreover, the order of these three stages can be interchanged leading to a total of six variations. The results indicate that whether temporal processing is performed before or after adaptation, the strategy that performs adaptation before spatial cooperation leads to smaller mean-square error. The additional temporal processing step is useful in combating perturbations due to noise over the communications links. We further describe an application in the context of distributed detection and provide computer simulations to illustrate and support the findings.

Proceedings ArticleDOI
18 Oct 2012
TL;DR: This work proposes a distributed LMS algorithm that achieves asymptotically unbiased estimates via diffusion adaptation and analyzes the performance of the proposed algorithm and provides computer experiments to illustrate its behavior.
Abstract: We study distributed least-mean square (LMS) estimation problems over adaptive networks, where nodes cooperatively work to estimate and track common parameters of an unknown system. We consider a scenario where the input and output response signals of the unknown system are both contaminated by measurement noise. In this case, if standard distributed estimation is performed without considering the effect of regression noise, then the resulting parameter estimates will be biased. To resolve this problem, we propose a distributed LMS algorithm that achieves asymptotically unbiased estimates via diffusion adaptation. We analyze the performance of the proposed algorithm and provide computer experiments to illustrate its behavior.

Proceedings ArticleDOI
04 Oct 2012
TL;DR: The mean-square-error performance of the diffusion strategy is analyzed and it is shown that, at steady-state, all nodes can be made to approach a Pareto-optimal solution.
Abstract: We consider solving multi-objective optimization problems in a distributed manner over a network of nodes. The problem is equivalent to optimizing a global cost that is the sum of individual components. Diffusion adaptation enables the nodes to cooperate locally through in-network processing in order to approach Pareto-optimality. We analyze the mean-square-error performance of the diffusion strategy and show that, at steady-state, all nodes can be made to approach a Pareto-optimal solution.

Journal ArticleDOI
TL;DR: A fast swarming approach is proposed, robust to random disturbances, that adapts its behavior with respect to the interference power perceived by every node, thus increasing the speed of convergence and improving the resource allocation capabilities.
Abstract: This paper proposes a distributed resource assignment strategy for cognitive networks mimicking a swarm foraging mechanism, assuming that the communication among the cognitive nodes is impaired by random link failures and quantization noise. Using results from stochastic approximation theory, we propose a swarm mechanism that converges almost surely to a final allocation even in the presence of imperfect communication scenarios. The theoretical findings are corroborated by numerical results showing that the only effect of the random link failures is to decrease the convergence rate of the algorithm. We propose then a fast swarming approach, robust to random disturbances, that adapts its behavior with respect to the interference power perceived by every node, thus increasing the speed of convergence and improving the resource allocation capabilities.

Journal ArticleDOI
TL;DR: This paper proposes low-complexity digital signal processing methods for estimating the jitter in real-time for direct downconversion receivers at high sampling rates and proposes adaptive compensation methods for the jitters.
Abstract: Clock timing jitter refers to random perturbations in the sampling time in analog-to-digital converters (ADCs). The perturbations are caused by circuit imperfections in the sampling clock. This paper analyzes the effect of sampling clock jitter on the acquired samples in the midst of random noise. We propose low-complexity digital signal processing methods for estimating the jitter in real-time for direct downconversion receivers at high sampling rates. We also propose adaptive compensation methods for the jitter and analyze the performance of the proposed techniques in some detail as well as through simulations.

Proceedings ArticleDOI
04 Oct 2012
TL;DR: A least mean-squares (LMS) diffusion strategy for sensor network applications where it is desired to estimate parameters of physical phenomena that vary over space using a set of basis functions to replace the space-variant parameters with space-invariant parameters.
Abstract: We develop a least mean-squares (LMS) diffusion strategy for sensor network applications where it is desired to estimate parameters of physical phenomena that vary over space. In particular, we consider a regression model with space-varying parameters that captures the system dynamics over time and space. We use a set of basis functions such as sinusoids or B-spline functions to replace the space-variant (local) parameters with space-invariant (global) parameters, and then apply diffusion adaptation to estimate the global representation. We illustrate the performance of the algorithm via simulations.

Proceedings ArticleDOI
04 Oct 2012
TL;DR: Diffusion strategies allow information to diffuse more thoroughly through the network, and this property has a favorable effect on the evolution of the network: diffusion networks reach lower mean-square deviation than consensus networks, and their mean- square stability is insensitive to the choice of the combination weights.
Abstract: Adaptive networks consist of a collection of nodes that interact with each other on a local level and diffuse information across the network to solve estimation and inference tasks in a distributed manner. In this work, we compare the performance of two distributed estimation strategies: diffusion and consensus. Diffusion strategies allow information to diffuse more thoroughly through the network. The analysis in the paper confirms that this property has a favorable effect on the evolution of the network: diffusion networks reach lower mean-square deviation than consensus networks, and their mean-square stability is insensitive to the choice of the combination weights. In contrast, consensus networks can become unstable even if all the individual nodes are mean-square stable; this does not occur for diffusion networks: stability of the individual nodes ensures stability of the diffusion network irrespective of the topology.

Proceedings ArticleDOI
17 Jun 2012
TL;DR: Simulation results support the findings that the MSE performance improves uniformly across the network relative to non-cooperative designs, including its transient and steady-state behavior.
Abstract: In this work, we consider a distributed beam coordination problem, where a collection of arrays are interconnected by a certain topology. The beamformers employ an adaptive diffusion strategy to compute the beamforming weight vectors by relying solely on cooperation with their local neighbors. We analyze the mean-square-error (MSE) performance of the proposed strategy, including its transient and steady-state behavior. Simulation results support the findings that the MSE performance improves uniformly across the network relative to non-cooperative designs.

Journal ArticleDOI
TL;DR: An algorithm that uses a Fourier Transform block (a typical component in spectrum sensing architectures) to estimate the sampling errors from the spurious sidebands is proposed and computer simulations are included to show that the proposed solution can remove the spuriousSidebands and improve the detection performance.
Abstract: In a nonideal PLL circuit, leakage of the reference signal into the control line produces spurious tones. When the distorted PLL signal is used as a sampling clock to an analog-to-digital converter (ADC), it creates spurious sidebands in the sampled data as well. In spectrum sensing applications, the presence of spurious sidebands can lead to false detection of signals in otherwise empty channels. To remove the spurious sidebands, we first estimate the distortions to the sampled data and then compensate the data in the digital domain. To reduce hardware complexity and computation cost, we propose an algorithm that uses a Fourier Transform block (a typical component in spectrum sensing architectures) to estimate the sampling errors from the spurious sidebands. We also analyze the effects of the spurious sidebands on spectrum sensing. Computer simulations are included to show that the proposed solution can remove the spurious sidebands and improve the detection performance.

Proceedings ArticleDOI
10 Jun 2012
TL;DR: This paper investigates the mean-square performance of adaptive diffusion algorithms in the presence of various sources of imperfect information exchanges and quantization errors, and reveals that link noise over the regression data modifies the dynamics of the network evolution, and leads to biased estimates in steady-state.
Abstract: Adaptive networks rely on in-network and collaborative processing among distributed agents to deliver enhanced performance in estimation and inference tasks. Information is exchanged among the nodes, usually over noisy links. This paper first investigates the mean-square performance of adaptive diffusion algorithms in the presence of various sources of imperfect information exchanges and quantization errors. Among other results, the analysis reveals that link noise over the regression data modifies the dynamics of the network evolution, and leads to biased estimates in steady-state. The analysis also reveals how the network mean-square performance is dependent on the combination weight matrices. We use these observations to show how the combination weights can be optimized and adapted. Simulation results illustrate the theoretical findings and match well with theory.

Proceedings ArticleDOI
12 Nov 2012
TL;DR: A fully-distributed stochastic-gradient strategy based on diffusion adaptation techniques is proposed, which shows that, for strongly convex risk functions, the excess-risk at every node decays at the rate of O(1/Ni), where N is the number of learners and i is the iteration index.
Abstract: We propose a fully-distributed stochastic-gradient strategy based on diffusion adaptation techniques. We show that, for strongly convex risk functions, the excess-risk at every node decays at the rate of O(1/Ni), where N is the number of learners and i is the iteration index. In this way, the distributed diffusion strategy, which relies only on local interactions, is able to achieve the same convergence rate as centralized strategies that have access to all data from the nodes at every iteration. We also show that every learner is able to improve its excess-risk in comparison to the non-cooperative mode of operation where each learner would operate independently of the other learners.

Proceedings ArticleDOI
01 Nov 2012
TL;DR: This work develops and study a procedure by which the entire network can be made to follow one objective or the other through a distributed and collaborative decision process.
Abstract: It is common for biological networks to encounter situations where agents need to decide between multiple options, such as deciding between moving towards one food source or another or between moving towards a new hive or another. In previous works, we developed several powerful diffusion strategies that allow agents to estimate a model of interest in an adaptive and distributed manner through a process of in-network collaboration and learning. In this work, we consider the situation in which the data observed by the agents may arise from two different distributions or models. We develop and study a procedure by which the entire network can be made to follow one objective or the other through a distributed and collaborative decision process.

Journal ArticleDOI
16 Feb 2012-PLOS ONE
TL;DR: This study demonstrates that the behavior of the p53-MDM2 gene network in individual cells can be tracked using adaptive filtering algorithms and the resulting time-variant models can approximate experimental measurements more accurately than time-invariant models.
Abstract: Biological systems are often treated as time-invariant by computational models that use fixed parameter values In this study, we demonstrate that the behavior of the p53-MDM2 gene network in individual cells can be tracked using adaptive filtering algorithms and the resulting time-variant models can approximate experimental measurements more accurately than time-invariant models Adaptive models with time-variant parameters can help reduce modeling complexity and can more realistically represent biological systems

Proceedings ArticleDOI
25 Mar 2012
TL;DR: An adaptive diffusion mechanism to optimize global cost functions in a distributed manner over a network of nodes in order to solve the desired optimization problem is derived.
Abstract: We derive an adaptive diffusion mechanism to optimize global cost functions in a distributed manner over a network of nodes. The cost function is assumed to consist of the sum of individual components, and diffusion adaptation is used to enable the nodes to cooperate locally through in-network processing in order to solve the desired optimization problem. We analyze the mean-square-error performance of the algorithm, including its transient and steady-state behavior. We illustrate one application in the context of least-mean-squares estimation for sparse vectors.

Proceedings ArticleDOI
10 Jun 2012
TL;DR: This work develops a decentralized adaptive strategy for throughput maximization over peer-to-peer (P2P) networks that can cope with changing network topologies, is robust to network disruptions, and does not rely on central processors.
Abstract: This work develops a decentralized adaptive strategy for throughput maximization over peer-to-peer (P2P) networks. The adaptive strategy can cope with changing network topologies, is robust to network disruptions, and does not rely on central processors. The algorithm is obtained as a special case of a more general diffusion strategy for the distributed solution of optimization problems with constraints. Simulation results illustrate how the proposed technique is competitive with other methods.

Proceedings ArticleDOI
01 Jan 2012
TL;DR: The proposed adaptive modeling approach can be a useful tool in the study of self-organizing behavior observed in other contexts in biology, including microbial pathogenesis, antibiotic resistance, embryonic development, tumor formation, etc.
Abstract: Using the transient interleukin (IL)-2 secretion of effector T helper (T eff ) cells as an example, we show that self-organizing multicellular behavior can be modeled and predicted by an adaptive gene network model. Incorporating an adaptation algorithm we established previously, we construct a network model that has the parameter values iteratively updated to cope with environmental change governed by diffusion and cell-cell interactions. In contrast to non-adaptive models, we find that the proposed adaptive model for individual T eff cells can generate transient IL-2 secretory behavior that is observed experimentally at the population level. The proposed adaptive modeling approach can be a useful tool in the study of self-organizing behavior observed in other contexts in biology, including microbial pathogenesis, antibiotic resistance, embryonic development, tumor formation, etc.

Proceedings ArticleDOI
28 May 2012
TL;DR: This work shows how the speed information can be exploited and incorporated into the design of the combination rules for mobile networks, and shows that the proposed combination rule leads to more effective information flow over networks of mobile agents.
Abstract: Collective motion is a remarkable phenomenon in biological systems. There have been several models in the literature to regenerate this type of motion, such as averaging consensus strategies where nodes continuously average the velocity vectors of their neighbors. While many models are able to generate forms of collective motion, they nevertheless neglect the important fact that the most informed nodes in a network tend to modulate their information into their speeds. In this work, we show how the speed information can be exploited and incorporated into the design of the combination rules for mobile networks. The analysis leads to a sigmoidal function construction, and the results show that the proposed combination rule leads to more effective information flow over networks of mobile agents.

Proceedings Article
01 Jan 2012
TL;DR: A distributed algorithm for online learning is proposed that is proved to guarantee a bounded excess risk and the bound can be made arbitrary small for sufficiently small step-sizes.
Abstract: We examine the problem of learning a set of parameters from a distributed dataset. We assume the datasets are collected by agents over a distributed ad-hoc network, and that the communication of the actual raw data is prohibitive due to either privacy constraints or communication constraints. We propose a distributed algorithm for online learning that is proved to guarantee a bounded excess risk and the bound can be made arbitrary small for sufficiently small step-sizes. We apply our framework to the expert advice problem where nodes learn the weights for the trained experts distributively.

Proceedings ArticleDOI
04 Oct 2012
TL;DR: It is proved that the new algorithm is stable and has better convergence properties than stand-alone learning for the case of doubly-stochastic mixing matrices.
Abstract: Discrete-time mobile adaptive networks have been successfully used to model self-organization in biological networks. We recently introduced a continuous-time adaptive diffusion strategy with the goal of better modeling physical phenomena governed by continuous-time dynamics. In the present paper we extend our previous work, proposing a new continuous-time diffusion estimation strategy that allows asymmetric mixing matrices. We prove that the new algorithm is stable and has better convergence properties than stand-alone learning for the case of doubly-stochastic mixing matrices.