Rate of convergence

About: Rate of convergence is a research topic. Over the lifetime, 31257 publications have been published within this topic receiving 795334 citations. The topic is also known as: convergence rate.

Distributed multi-agent optimization with state-dependent communication

Abstract: We study distributed algorithms for solving global optimization problems in which the objective function is the sum of local objective functions of agents and the constraint set is given by the intersection of local constraint sets of agents. We assume that each agent knows only his own local objective function and constraint set, and exchanges information with the other agents over a randomly varying network topology to update his information state. We assume a state-dependent communication model over this topology: communication is Markovian with respect to the states of the agents and the probability with which the links are available depends on the states of the agents. We study a projected multi-agent subgradient algorithm under state-dependent communication. The state-dependence of the communication introduces significant challenges and couples the study of information exchange with the analysis of subgradient steps and projection errors. We first show that the multi-agent subgradient algorithm when used with a constant stepsize may result in the agent estimates to diverge with probability one. Under some assumptions on the stepsize sequence, we provide convergence rate bounds on a “disagreement metric” between the agent estimates. Our bounds are time-nonhomogeneous in the sense that they depend on the initial starting time. Despite this, we show that agent estimates reach an almost sure consensus and converge to the same optimal solution of the global optimization problem with probability one under different assumptions on the local constraint sets and the stepsize sequence.

Time-varying sliding modes for second-order systems

Abstract: Three types of optimal, continuously time-varying sliding mode for robust control of second-order uncertain dynamic systems subject to input constraint are presented. Two of the modes incorporate straight sliding lines, and the third uses the so-called terminal slider, that is a curve that guarantees system error convergence to zero in finite time. At first, all three lines adapt themselves to the initial conditions of the system, and afterwards they move in such a way that, for each of them, the integral of the absolute value of the systems error is minimised over the whole period of the control action. By this means, insensitivity of the system to external disturbances and parameter uncertainties is guaranteed from the very beginning of the proposed control action, and the system error convergence rate can be increased. Performance of the three control algorithms is compared, and the Lyapunov theory is used to prove the existence of a sliding mode on the lines.

The Asymptotic Convergence-Rate of Q-learning

Abstract: In this paper we show that for discounted MDPs with discount factor γ > 1/2 the asymptotic rate of convergence of Q-learning is O(1/tR(1-γ)) if R(1 - γ) 0, where pmin and pmax now become the minimum and maximum state-action occupation frequencies corresponding to the stationary distribution.

Convergence of generalized proximal point algorithms

Abstract: Weak and strong convergence for some generalized proximal point algorithms are proved. These algorithms include the Eckstein and Bertsekas generalized proximal point algorithm, a contraction-proximal point algorithm, and inexact proximal point algorithms. Convergence rate is also considered.