This paper presents a converse Lyapunov function theorem motivated by robust control analysis and design. Our result is based upon, but generalizes, various aspects of well-known classical theorems. In a unified and natural manner, it (1) allows arbitrary bounded time-varying parameters in the system description, (2) deals with global asymptotic stability, (3) results in smooth (infinitely differentiable) Lyapunov functions, and (4) applies to stability with respect to not necessarily compact invariant sets.

A Smooth Converse Lyapunov Theorem for Robust Stability

We present a perfect simulation algorithm for stationary processes indexed by $\mathbb{Z}$, with summable memory decay. Depending on the decay, we construct the process on finite or semi-infinite intervals, explicitly from an i.i.d. uniform sequence. Even though the process has infinite memory, its value at time 0 depends only on a finite, but random, number of these uniform variables. The algorithm is based on a recent regenerative construction of these measures by Ferrari, Maass, Martinez and Ney. As applications, we discuss the perfect simulation of binary autoregressions and Markov chains on the unit interval.

/pdf/processes-with-long-memory-regenerative-construction-and-31wau1mh0e.pdf

Processes with long memory: Regenerative construction and perfect simulation

Consider a sample Zi,...,Z„ of i.i.d r.v.'s with tail behavior P{Zi > z) = r(z)exp(—Äz), where r( ) is an (unknown) regularly varying function as z —• oo, and R is a constant. Least squares estimators are proposed here for the problem of estimating the exponential tail coefiicient R. We mainly prove consistency of the proposed estimators. Some Simulation results are presented as well in order to illustrate the finite sample behavior. Herein, adaptive methods are suggested to determine the number k = k„ of upper order statistics, which should be taken into account for estimation. Without loss of generality we shall assume positivity of the random variables Zi

On least squares estimates of an exponential tail coefficient

The general multi-armed bandit problem is reformulated and solved as a control problem over a partially ordered set. The approach taken provides a technically convenient framework for bandit-like problems. It also adds insight to the structure of strategies over partially ordered sets.

Discrete multi-armed bandits and multi-parameter processes

We consider a one-hop wireless system with a small number of delay constrained users and a larger number of users without delay constraints. We develop a scheduling algorithm that reacts to time varying channels and maximizes throughput utility (to within a desired proximity), stabilizes all queues, and satisfies the delay constraints. The problem is solved by reducing the constrained optimization to a set of weighted stochastic shortest path problems, which act as natural generalizations of max-weight policies to Markov decision networks. We also present approximation results for the corresponding shortest path problems, and discuss the additional complexity and delay incurred as compared to systems without delay constraints. The solution technique is general and applies to other constrained stochastic decision problems.

/pdf/dynamic-markov-decision-policies-for-delay-constrained-4sn8jw3e4h.pdf

Dynamic Markov Decision Policies for Delay Constrained Wireless Scheduling

General control systems

On the Oberwolfach-conference on risk theory in September, 1984, Professor Buhlmann proposed to develop the statistics for risk theory. The aim of this paper is to deliver a contribution to this project. It shows how stochastic approximation methods may be used to estimate the so-called adjustment coefficient in risk theory.

On the estimation of the adjustment coefficient in risk theory by means of stochastic approximation procedures

SUMMARY The Robbins-Monro stochastic approximation procedure is modified so as to be applicable in the presence of delayed observations; asymptotic properties of the modified procedure are investigated.

Ulrich Herkenrath

Papers

General control systems

On the estimation of the adjustment coefficient in risk theory by means of stochastic approximation procedures

Stochastic approximation with delayed observations

On a general class of absorbing-barrier learning algorithms

Mathematical Learning Models — Theory and Algorithms