Showing papers on "Convergence (routing) published in 2017"
Posted Content•
TL;DR: A convergence analysis for SGD is provided on a rich subset of two-layer feedforward networks with ReLU activations characterized by a special structure called "identity mapping" that proves that, if input follows from Gaussian distribution, with standard $O(1/\sqrt{d})$ initialization of the weights, SGD converges to the global minimum in polynomial number of steps.
Abstract: In recent years, stochastic gradient descent (SGD) based techniques has become the standard tools for training neural networks. However, formal theoretical understanding of why SGD can train neural networks in practice is largely missing.
In this paper, we make progress on understanding this mystery by providing a convergence analysis for SGD on a rich subset of two-layer feedforward networks with ReLU activations. This subset is characterized by a special structure called "identity mapping". We prove that, if input follows from Gaussian distribution, with standard $O(1/\sqrt{d})$ initialization of the weights, SGD converges to the global minimum in polynomial number of steps. Unlike normal vanilla networks, the "identity mapping" makes our network asymmetric and thus the global minimum is unique. To complement our theory, we are also able to show experimentally that multi-layer networks with this mapping have better performance compared with normal vanilla networks.
Our convergence theorem differs from traditional non-convex optimization techniques. We show that SGD converges to optimal in "two phases": In phase I, the gradient points to the wrong direction, however, a potential function $g$ gradually decreases. Then in phase II, SGD enters a nice one point convex region and converges. We also show that the identity mapping is necessary for convergence, as it moves the initial point to a better place for optimization. Experiment verifies our claims.
440 citations
TL;DR: It was shown by the results on two problems from practice that the proposed vector angle-based evolutionary algorithm significantly outperforms its competitors in terms of both the convergence and diversity of the obtained solution sets.
Abstract: Taking both convergence and diversity into consideration, this paper suggests a vector angle-based evolutionary algorithm for unconstrained (with box constraints only) many-objective optimization problems. In the proposed algorithm, the maximum-vector-angle-first principle is used in the environmental selection to guarantee the wideness and uniformity of the solution set. With the help of the worse-elimination principle, worse solutions in terms of the convergence (measured by the sum of normalized objectives) are allowed to be conditionally replaced by other individuals. Therefore, the selection pressure toward the Pareto-optimal front is strengthened. The proposed method is compared with other four state-of-the-art many-objective evolutionary algorithms on a number of unconstrained test problems with up to 15 objectives. The experimental results have shown the competitiveness and effectiveness of our proposed algorithm in keeping a good balance between convergence and diversity. Furthermore, it was shown by the results on two problems from practice (with irregular Pareto fronts) that our method significantly outperforms its competitors in terms of both the convergence and diversity of the obtained solution sets. Notably, the new algorithm has the following good properties: 1) it is free from a set of supplied reference points or weight vectors; 2) it has less algorithmic parameters; and 3) the time complexity of the algorithm is low. Given both good performance and nice properties, the suggested algorithm could be an alternative tool when handling optimization problems with more than three objectives.
299 citations
TL;DR: If each agent is asymptotically null controllable with bounded controls and the interaction topology described by a signed digraph is structurally balanced and contains a spanning tree, then the semi-global bipartite consensus can be achieved for the linear multiagent system by a linear feedback controller with the control gain being designed via the low gain feedback technique.
Abstract: The bipartite consensus problem for a group of homogeneous generic linear agents with input saturation under directed interaction topology is examined. It is established that if each agent is asymptotically null controllable with bounded controls and the interaction topology described by a signed digraph is structurally balanced and contains a spanning tree, then the semi-global bipartite consensus can be achieved for the linear multiagent system by a linear feedback controller with the control gain being designed via the low gain feedback technique. The convergence analysis of the proposed control strategy is performed by means of the Lyapunov method which can also specify the convergence rate. At last, the validity of the theoretical findings is demonstrated by two simulation examples.
272 citations
Proceedings Article•
01 May 2017TL;DR: In this article, the authors provide a convergence analysis for SGD on a rich subset of two-layer feedforward networks with ReLU activations, characterized by a special structure called "identity mapping".
Abstract: In recent years, stochastic gradient descent (SGD) based techniques has become the standard tools for training neural networks. However, formal theoretical understanding of why SGD can train neural networks in practice is largely missing. In this paper, we make progress on understanding this mystery by providing a convergence analysis for SGD on a rich subset of two-layer feedforward networks with ReLU activations. This subset is characterized by a special structure called "identity mapping". We prove that, if input follows from Gaussian distribution, with standard $O(1/\sqrt{d})$ initialization of the weights, SGD converges to the global minimum in polynomial number of steps. Unlike normal vanilla networks, the "identity mapping" makes our network asymmetric and thus the global minimum is unique. To complement our theory, we are also able to show experimentally that multi-layer networks with this mapping have better performance compared with normal vanilla networks. Our convergence theorem differs from traditional non-convex optimization techniques. We show that SGD converges to optimal in "two phases": In phase I, the gradient points to the wrong direction, however, a potential function $g$ gradually decreases. Then in phase II, SGD enters a nice one point convex region and converges. We also show that the identity mapping is necessary for convergence, as it moves the initial point to a better place for optimization. Experiment verifies our claims.
263 citations
TL;DR: A model-free solution to the H ∞ control of linear discrete-time systems is presented that employs off-policy reinforcement learning (RL) to solve the game algebraic Riccati equation online using measured data along the system trajectories.
157 citations
TL;DR: A novel discrete-time deterministic deterministic inline-formula-learning algorithm is developed and the convergence criterion for the discounted case is established, and the iterative control law of the developed algorithm is simplified.
Abstract: In this paper, a novel discrete-time deterministic $ Q$ -learning algorithm is developed. In each iteration of the developed $ Q$ -learning algorithm, the iterative $ Q$ function is updated for all the state and control spaces, instead of updating for a single state and a single control in traditional $ Q$ -learning algorithm. A new convergence criterion is established to guarantee that the iterative $ Q$ function converges to the optimum, where the convergence criterion of the learning rates for traditional $ Q$ -learning algorithms is simplified. During the convergence analysis, the upper and lower bounds of the iterative $ Q$ function are analyzed to obtain the convergence criterion, instead of analyzing the iterative $ Q$ function itself. For convenience of analysis, the convergence properties for undiscounted case of the deterministic $ Q$ -learning algorithm are first developed. Then, considering the discounted factor, the convergence criterion for the discounted case is established. Neural networks are used to approximate the iterative $ Q$ function and compute the iterative control law, respectively, for facilitating the implementation of the deterministic $ Q$ -learning algorithm. Finally, simulation results and comparisons are given to illustrate the performance of the developed algorithm.
157 citations
TL;DR: In this article, the authors consider the problem of distributed learning where a network of agents collectively aim to agree on a hypothesis that best explains a set of distributed observations of conditionally independent random processes.
Abstract: We consider the problem of distributed learning , where a network of agents collectively aim to agree on a hypothesis that best explains a set of distributed observations of conditionally independent random processes. We propose a distributed algorithm and establish consistency, as well as a nonasymptotic, explicit, and geometric convergence rate for the concentration of the beliefs around the set of optimal hypotheses. Additionally, if the agents interact over static networks, we provide an improved learning protocol with better scalability with respect to the number of nodes in the network.
155 citations
Posted Content•
TL;DR: A collaborative multi-agent reinforcement learning (MARL) approach is employed to jointly train the router and function blocks of a routing network, a kind of self-organizing neural network consisting of a router and a set of one or more function blocks.
Abstract: Multi-task learning (MTL) with neural networks leverages commonalities in tasks to improve performance, but often suffers from task interference which reduces the benefits of transfer. To address this issue we introduce the routing network paradigm, a novel neural network and training algorithm. A routing network is a kind of self-organizing neural network consisting of two components: a router and a set of one or more function blocks. A function block may be any neural network - for example a fully-connected or a convolutional layer. Given an input the router makes a routing decision, choosing a function block to apply and passing the output back to the router recursively, terminating when a fixed recursion depth is reached. In this way the routing network dynamically composes different function blocks for each input. We employ a collaborative multi-agent reinforcement learning (MARL) approach to jointly train the router and function blocks. We evaluate our model against cross-stitch networks and shared-layer baselines on multi-task settings of the MNIST, mini-imagenet, and CIFAR-100 datasets. Our experiments demonstrate a significant improvement in accuracy, with sharper convergence. In addition, routing networks have nearly constant per-task training cost while cross-stitch networks scale linearly with the number of tasks. On CIFAR-100 (20 tasks) we obtain cross-stitch performance levels with an 85% reduction in training time.
146 citations
TL;DR: A class of deterministic nonlinear models for the propagation of infectious diseases over contact networks with strongly-connected topologies is reviewed and novel results for transient behavior, threshold conditions, stability properties, and asymptotic convergence are proposed.
131 citations
TL;DR: Two different control protocols, namely, full-state feedback and static output-feedback, are designed based on internal model principles based oninternal model principles to guarantee the convergence of the output of each follower to the dynamic convex hull spanned by the outputs of leaders.
Abstract: This paper studies the output containment control of linear heterogeneous multi-agent systems, where the system dynamics and even the state dimensions can generally be different. Since the states can have different dimensions, standard results from state containment control do not apply. Therefore, the control objective is to guarantee the convergence of the output of each follower to the dynamic convex hull spanned by the outputs of leaders. This can be achieved by making certain output containment errors go to zero asymptotically. Based on this formulation, two different control protocols, namely, full-state feedback and static output-feedback, are designed based on internal model principles. Sufficient local conditions for the existence of the proposed control protocols are developed in terms of stabilizing the local followers’ dynamics and satisfying a certain ${H_\infty }$ criterion. Unified design procedures to solve the proposed two control protocols are presented by formulation and solution of certain local state-feedback and static output-feedback problems, respectively. Numerical simulations are given to validate the proposed control protocols.
126 citations
TL;DR: This work proposes an extragradient method with stepsizes bounded away from zero for stochastic variational inequalities requiring only pseudomonotonicity, and provides convergence and complexity analysis.
Abstract: We propose an extragradient method with stepsizes bounded away from zero for stochastic variational inequalities requiring only pseudomonotonicity. We provide convergence and complexity analysis, a...
TL;DR: This work proves the instability of the first distributed protocol to address the winner-take-all (WTA) problem on networks, and global convergence to the WTA solution via Lyapunov theory.
Abstract: This paper is concerned with the winner-take-all (WTA) problem on networks. We propose the first distributed protocol to address this problem dynamically. This protocol features strong nonlinearity. Theoretical analysis reveals that it contains invariant quantities, symmetric solutions, and multiple equilibrium points. By leveraging these properties, this work proves the instability of its non-WTA solutions, and global convergence to the WTA solution via Lyapunov theory. Two simulations over networks with 10 and 200 nodes, respectively, are conducted. Simulation results have well verified the theoretical conclusions drawn in this paper.
TL;DR: An innovative parameter-adaptive strategy for ant colony optimization (ACO) algorithms based on controlling the convergence trajectory in decision space to follow any prespecified path, aimed at finding the best possible solution within a given, and limited, computational budget.
Abstract: Evolutionary algorithms and other meta-heuristics have been employed widely to solve optimization problems in many different fields over the past few decades. Their performance in finding optimal solutions often depends heavily on the parameterization of the algorithm’s search operators, which affect an algorithm’s balance between search diversification and intensification. While many parameter-adaptive algorithms have been developed to improve the searching ability of meta-heuristics, their performance is often unsatisfactory when applied to real-world problems. This is, at least in part, because available computational budgets are often constrained in such settings due to the long simulation times associated with objective function and/or constraint evaluation, thereby preventing convergence of existing parameter-adaptive algorithms. To this end, this paper proposes an innovative parameter-adaptive strategy for ant colony optimization (ACO) algorithms based on controlling the convergence trajectory in decision space to follow any prespecified path, aimed at finding the best possible solution within a given, and limited, computational budget. The utility of the proposed convergence-trajectory controlled ACO (ACO $_{\mathbf{CTC}}$ ) algorithm is demonstrated using six water distribution system design problems (WDSDPs, a difficult type of combinatorial problem in water resources) with varying complexity. The results show that the proposed ACO $_{\mathbf{CTC}}$ successfully enables the specified convergence trajectories to be followed by automatically adjusting the algorithm’s parameter values. Different convergence trajectories significantly affect the algorithm’s final performance (solution quality). The trajectory with a slight bias toward diversification in the first half and more emphasis on intensification during the second half of the search exhibits substantially improved performance compared to the best available ACO variant with the best parameterization (no convergence control) for all WDSDPs and computational scenarios considered. For the two large-scale WDSDPs, new best-known solutions are found by the proposed ACO $_{\mathbf{CTC}}$ .
TL;DR: The analysis indicates that the parameter estimates given by the proposed algorithms converge to their true values under the persistent excitation conditions.
TL;DR: The linear rate convergence of the alternating direction method of multipliers (ADMM) for solving linearly constrained convex composite optimization problems is proved and the usefulness of the obtained results when applied to two- and multi-block convex quadratic (semidefinite) programming.
Abstract: In this paper, we aim to prove the linear rate convergence of the alternating direction method of multipliers (ADMM) for solving linearly constrained convex composite optimization problems. Under a mild calmness condition, which holds automatically for convex composite piecewise linear-quadratic programming, we establish the global Q-linear rate of convergence for a general semi-proximal ADMM with the dual step-length being taken in (0, (1+51/2)/2). This semi-proximal ADMM, which covers the classic one, has the advantage to resolve the potentially nonsolvability issue of the subproblems in the classic ADMM and possesses the abilities of handling the multi-block cases efficiently. We demonstrate the usefulness of the obtained results when applied to two- and multi-block convex quadratic (semidefinite) programming.
16 Jan 2017
TL;DR: In this paper, it was shown that the complexity of leader election and majority election can be reduced to O(log log n) and O(n/poly logn) expected time, respectively, by using a super-constant number of states per node.
Abstract: Population protocols are a popular model of distributed computing, in which randomly-interacting agents with little computational power cooperate to jointly perform computational tasks. Inspired by developments in molecular computation, and in particular DNA computing, recent algorithmic work has focused on the complexity of solving simple yet fundamental tasks in the population model, such as leader election (which requires convergence to a single agent in a special "leader" state), and majority (in which agents must converge to a decision as to which of two possible initial states had higher initial count). Known results point towards an inherent trade-off between the time complexity of such algorithms, and the space complexity, i.e. size of the memory available to each agent.In this paper, we explore this trade-off and provide new upper and lower bounds for majority and leader election. First, we prove a unified lower bound, which relates the space available per node with the time complexity achievable by a protocol: for instance, our result implies that any protocol solving either of these tasks for n agents using O(log log n) states must take Ω(n/polylogn) expected time. This is the first result to characterize time complexity for protocols which employ super-constant number of states per node, and proves that fast, poly-logarithmic running times require protocols to have relatively large space costs.On the positive side, we give algorithms showing that fast, poly-logarithmic convergence time can be achieved using O(log2n) space per node, in the case of both tasks. Overall, our results highlight a time complexity separation between O (log log n) and Θ(log2n) state space size for both majority and leader election in population protocols, and introduce new techniques, which should be applicable more broadly.
TL;DR: Stochastic numerical computing approach is developed by applying artificial neural networks (ANNs) to compute the solution of Lane–Emden type boundary value problems arising in thermodynamic studies of the spherical gas cloud model.
Abstract: In the present study, stochastic numerical computing approach is developed by applying artificial neural networks (ANNs) to compute the solution of Lane–Emden type boundary value problems arising in thermodynamic studies of the spherical gas cloud model. ANNs are used in an unsupervised manner to construct the energy function of the system model. Strength of efficient local optimization procedures based on active-set (AS), interior-point (IP) and sequential quadratic programming (SQP) algorithms is used to optimize the energy functions. The performance of all three design methodologies ANN-AS, ANN-IP and ANN-SQP is evaluated on different nonlinear singular systems. The effectiveness of the proposed schemes in terms of accuracy and convergence is established from the results of statistical indicators.
Posted Content•
TL;DR: In this paper, a class of deterministic nonlinear models for the propagation of infectious diseases over contact networks with strongly-connected topologies is presented. And the authors provide a comprehensive nonlinear analysis of equilibria, stability properties, convergence, monotonicity, positivity, and threshold conditions.
Abstract: In this work we review a class of deterministic nonlinear models for the propagation of infectious diseases over contact networks with strongly-connected topologies. We consider network models for susceptible-infected (SI), susceptible-infected-susceptible (SIS), and susceptible-infected-recovered (SIR) settings. In each setting, we provide a comprehensive nonlinear analysis of equilibria, stability properties, convergence, monotonicity, positivity, and threshold conditions. For the network SI setting, specific contributions include establishing its equilibria, stability, and positivity properties. For the network SIS setting, we review a well-known deterministic model, provide novel results on the computation and characterization of the endemic state (when the system is above the epidemic threshold), and present alternative proofs for some of its properties. Finally, for the network SIR setting, we propose novel results for transient behavior, threshold conditions, stability properties, and asymptotic convergence. These results are analogous to those well-known for the scalar case. In addition, we provide a novel iterative algorithm to compute the asymptotic state of the network SIR system.
TL;DR: An efficient computing technique has been developed for the solution of fractional order systems governed with initial value problems (IVPs) of the BagleyTorvik equations using fractional neural networks (FNNs) optimized with interior point algorithms (IPAs).
TL;DR: In this article, the authors analyzed the deterministic incremental aggregated gradient method for minimizing a finite sum of smooth functions where the sum is strongly convex and showed that this deterministic algorithm has global linear convergence and characterized the convergence rate.
Abstract: Motivated by applications to distributed optimization over networks and large-scale data processing in machine learning, we analyze the deterministic incremental aggregated gradient method for minimizing a finite sum of smooth functions where the sum is strongly convex. This method processes the functions one at a time in a deterministic order and incorporates a memory of previous gradient values to accelerate convergence. Empirically it performs well in practice; however, no theoretical analysis with explicit rate results was previously given in the literature to our knowledge, in particular most of the recent efforts concentrated on the randomized versions. In this paper, we show that this deterministic algorithm has global linear convergence and we characterize the convergence rate. We also consider an aggregated method with momentum and demonstrate its linear convergence. Our proofs rely on a careful choice of a Lyapunov function that offers insight into the algorithm's behavior and simplifies the pro...
TL;DR: It is proved that from any initial state, the state of the proposed neural network reaches the feasible region in finite time and stays there thereafter and is convergent to an optimal solution of the related problem.
Abstract: Pseudoconvex optimization problem, as an important nonconvex optimization problem, plays an important role in scientific and engineering applications. In this paper, a recurrent one-layer neural network is proposed for solving the pseudoconvex optimization problem with equality and inequality constraints. It is proved that from any initial state, the state of the proposed neural network reaches the feasible region in finite time and stays there thereafter. It is also proved that the state of the proposed neural network is convergent to an optimal solution of the related problem. Compared with the related existing recurrent neural networks for the pseudoconvex optimization problems, the proposed neural network in this paper does not need the penalty parameters and has a better convergence. Meanwhile, the proposed neural network is used to solve three nonsmooth optimization problems, and we make some detailed comparisons with the known related conclusions. In the end, some numerical examples are provided to illustrate the effectiveness of the performance of the proposed neural network.
TL;DR: A non-singular fixed-time distributed control protocol for second-order multi-agent systems is designed, which only requires one-hop information of the neighbours without the global topology information and has the advantage of fast convergence performance both in the reaching phase and sliding phase.
Abstract: This paper is devoted to the fixed-time consensus tracking control for second-order multi-agent systems with bounded input uncertainties under a weighted directed topology. Firstly, a novel non-singular fixed-time fast terminal sliding mode (NFFTSM) surface with bounded convergence time in regardless of the initial states is designed, and the explicit expression of the settling time is provided. Fair and unprejudiced comparisons show that the proposed NFFTSM has faster convergence performance than most typical terminal sliding modes in the existed results. Subsequently, by employing the proposed NFFTSM, a non-singular fixed-time distributed control protocol for second-order multi-agent systems is designed, which only requires one-hop information of the neighbours without the global topology information and has the advantage of fast convergence performance both in the reaching phase and sliding phase. Rigorous proofs show that the fixed-time consensus tracking control for second-order multi-agent systems can be guaranteed by the proposed distributed control protocol. Finally, numerical simulations are performed to demonstrate the effectiveness of the proposed control scheme.
TL;DR: A two-grid convergence theory for the parallel-in-time scheme known as multigrid reduction in time (MGRIT), as it is implemented in the open-source package XBraid, and presents a two-level MGRIT convergence analysis for linear problems where the spatial discretization matrix can be diagonalized.
Abstract: In this paper we develop a two-grid convergence theory for the parallel-in-time scheme known as multigrid reduction in time (MGRIT), as it is implemented in the open-source package [XBraid: Parallel Multigrid in Time, http://llnl.gov/casc/xbraid]. MGRIT is a scalable and multilevel approach to parallel-in-time simulations that nonintrusively uses existing time-stepping schemes, and in a specific two-level setting it is equivalent to the widely known parareal algorithm. The goal of this paper is twofold. First, we present a two-level MGRIT convergence analysis for linear problems where the spatial discretization matrix can be diagonalized, and then apply this analysis to our two basic model problems, the heat equation and the advection equation. One important assumption is that the coarse and fine time-grid propagators can be diagaonalized by the same set of eigenvectors, which is often the case when the same spatial discretization operator is used on the coarse and fine time grids. In many cases, the MGRI...
TL;DR: The proposed DMHE for a class of two-time-scale nonlinear systems described in the framework of singularly perturbed systems and its application to a chemical process example demonstrates its applicability and effectiveness.
TL;DR: This paper considers the discrete-time version of Altafini's model for opinion dynamics in which the interaction among a group of agents is described by a time-varying signed digraph, and shows that a certain type of two-clustering will be reached exponentially fast for almost all initial conditions if, and only if, the sequence ofsigned digraphs is repeatedly jointly structurally balanced corresponding to that type of three-clusters.
Abstract: This paper considers the discrete-time version of Altafini's model for opinion dynamics in which the interaction among a group of agents is described by a time-varying signed digraph. Prompted by an idea from [3] , exponential convergence of the system is studied using a graphical approach. Necessary and sufficient conditions for exponential convergence with respect to each possible type of limit states are provided. Specifically, under the assumption of repeatedly jointly strong connectivity, it is shown that 1) a certain type of two-clustering will be reached exponentially fast for almost all initial conditions if, and only if, the sequence of signed digraphs is repeatedly jointly structurally balanced corresponding to that type of two-clustering; 2) the system will converge to zero exponentially fast for all initial conditions if, and only if, the sequence of signed digraphs is repeatedly jointly structurally unbalanced. An upper bound on the convergence rate is provided. The results are also extended to the continuous-time Altafini model.
TL;DR: This paper revisits the consensus-based projected subgradient algorithm and provides a systematical analysis to derive the asymptotic upper bound of convergence rates in terms of the optimum residual, and select the best step sizes accordingly.
TL;DR: A radial space division based evolutionary algorithm for many-objective optimization, where the solutions in high-dimensional objective space are projected into the grid divided 2-dimensional radial space for diversity maintenance and convergence enhancement.
TL;DR: Two identification algorithms are developed in order to identify time-varying parameters in a finite time or prescribed time (fixed-time) and convergence proofs are based on a notion of finite-time stability over finite intervals of time, i.e., short-finite- time stability, homogeneity for time- varying systems, and Lyapunov-based approach.
Abstract: In this paper, the problem of time-varying parameter identification is studied. To this aim, two identification algorithms are developed in order to identify time-varying parameters in a finite time or prescribed time (fixed-time). The convergence proofs are based on a notion of finite-time stability over finite intervals of time, i.e., short-finite-time stability, homogeneity for time-varying systems, and Lyapunov-based approach. The results are obtained under injectivity of the regressor term, which is related to the classical identifiability condition. The case of bounded disturbances (noise of measurements) is analyzed for both algorithms. Simulation results illustrate the feasibility of the proposed algorithms.
TL;DR: A protocol for the average consensus problem on any fixed undirected graph whose convergence time scales linearly in the total number nodes $n$ and has error which is $O(L \sqrt{n/T})$.
Abstract: We describe a protocol for the average consensus problem on any fixed undirected graph whose convergence time scales linearly in the total number nodes $n$. The protocol relies only on nearest-neig...
TL;DR: In this article, the generalized power method (GPM) was shown to be an optimal solution to the Gaussian noise problem and can be found with high probability using semidefinite programming (SDP).
Abstract: An estimation problem of fundamental interest is that of phase (or angular) synchronization, in which the goal is to recover a collection of phases (or angles) using noisy measurements of relative phases (or angle offsets). It is known that in the Gaussian noise setting, the maximum likelihood estimator (MLE) is an optimal solution to a nonconvex quadratic optimization problem and can be found with high probability using semidefinite programming (SDP), provided that the noise power is not too large. In this paper, we study the estimation and convergence performance of a recently proposed low-complexity alternative to the SDP-based approach, namely, the generalized power method (GPM). Our contribution is twofold. First, we show that the sequence of estimation errors associated with the GPM iterates is bounded above by a decreasing sequence. As a corollary, we show that all iterates achieve an estimation error that is on the same order as that of an MLE. Our result holds under the least restrictive assumpti...