Showing papers in "Advances in Applied Probability in 1995"

Sample mean based index policies by O(log n) regret for the multi-armed bandit problem

Abstract: We consider a non-Bayesian infinite horizon version of the multi-armed bandit problem with the objective of designing simple policies whose regret increases sldwly with time. In their seminal work on this problem, Lai and Robbins had obtained a O(logn) lower bound on the regret with a constant that depends on the KullbackLeibler number. They also constructed policies for some specific families of probability distributions (including exponential families) that achieved the lower bound. In this paper we construct index policies that depend on the rewards from each arm only through their sample mean. These policies are computationally much simpler and are also applicable much more generally. They achieve a O(logn) regret with a constant that is also based on the Kullback-Leibler number. This constant turns out to be optimal for one-parameter exponential families; however, in general it is derived from the optimal one via a 'contraction' principle. Our results rely entirely on a few key lemmas from the theory of large deviations.

Probability metrics and recursive algorithms

Abstract: It is shown by means of several examples that probability metrics are a useful tool to study the asymptotic behaviour of (stochastic) recursive algorithms. The basic idea of this approach is to find a 'suitable' probability metric which yields contraction properties of the transformations describing the limits of the algorithm. In order to demonstrate the wide range of applicability of this contraction method we investigate examples from various fields, some of which have already been analysed in the literature.

Boundary corrections for the expected Euler characteristic of excursion sets of random fields, with an application to astrophysics

Abstract: Certain images arising in astrophysics and medicine are modelled as smooth random fields inside a fixed region, and experimenters are interested in the number of 'peaks', or more generally, the topological structure of 'hot-spots' present in such an image. This paper studies the Euler characteristic of the excursion set of a random field ; the excursion set is the set of points where the image exceeds a fixed threshold, and the Euler characteristic counts the number of connected components in the excursion set minus the number of 'holes'. For high thresholds the Euler characteristic is a measure of the number of peaks. The geometry of excursion sets has been studied by Adler (1981) who gives the expectation of two excursion set characteristics, called the DT (differential topology) and IG (integral geometry) characteristics, which equal the Euler characteristic provided the excursion set does not touch the boundary of the region. Worsley (1995) finds a boundary correction which gives the expectation of the Euler characteristic itself in two and three dimensions. The proof uses a representation of the Euler characteristic given by Hadwiger (1959). The purpose of this paper is to give a general result for any number of dimensions. The proof takes a different approach and uses a representation from Morse theory. Results are applied to the recently discovered anomalies in the cosmic microwave background radiation, thought to be the remnants of the creation of the universe.

Likelihood ratio gradient estimation for stochastic recursions

Abstract: In this paper, we develop mathematical machinery for verifying that a broad class of general state space Markov chains reacts smoothly to certain types of perturbations in the underlying transition structure. Our main result provides conditions under which the stationary probability measure of an ergodic Harris-recurrent Markov chain is differentiable in a certain strong sense. The approach is based on likelihood ratio ‘change-of-measure' arguments, and leads directly to a ‘likelihood ratio gradient estimator' that can be computed numerically.

An algorithm to compute blocking probabilities in multi-rate multi-class multi-resource loss models

Abstract: In this paper we consider a family of product-form loss models, including loss networks (or circuit-switched communication networks) and a class of resource-sharing models. There can be multiple classes of requests for multiple resources. Requests arrive according to independent Poisson processes. The requests can be for multiple units in each resource (the multi-rate case, e.g. several circuits on a trunk). There can be upper-limit and guaranteed-minimum sharing policies as well as the standard complete-sharing policy. If all the requirements of a request cannot be met upon arrival, then the request is blocked and lost. We develop an algorithm for computing the (exact) steady-state blocking probability of each class and other steady state descriptions in these loss models. The algorithm is based on numerically inverting generating functions of the normalization constants. In a previous paper we introduced this approach to product-form models and developed a full algorithm for a class of closed queueing networks. The inversion algorithm promises to be even more useful for loss models than for closed queueing networks because fewer alternative algorithms are available for loss models. Indeed, for many loss models with sharing policies other than traditional complete sharing, our algorithm is the first effective algorithm. Unlike some recursive algorithms, our algorithm has a low storage requirement. To treat the loss models here, we derive the generating functions of the normalization constants and develop a new scaling algorithm especially tailored to the loss models. In general, the computational complexity grows exponentially in the number of resources, but the computation can often be reduced dramatically by exploiting conditional decomposition based on special structure and by appropriately truncating large finite sums. We illustrate our numerical inversion algorithm by applying it to several examples. To validate our algorithm on small models, we also develop a direct algorithm. The direct algorithm itself is of interest, because it tends to be more efficient when the number of resources is large, but the number of request classes is small. Furthermore, it also allows a form of conditional decomposition based on special structure.

Analysis of an M/G/1 Queue with Two Types of Impatient Units

Abstract: This paper deals with a service system in which the processor must serve two types of impatient units. In the case of blocking, the first type units leave the system whereas the second type units enter a pool and wait to be processed later. We develop an exhaustive analysis of the system including embedded Markov chain, fundamental period and various classical stationary probability distributions. More specific performance measures, such as the number of lost customers and other quantities, are also considered. The mathematical analysis of the model is based on the theory of Markov renewal processes, in Markov chains of M/G/1 type and in expressions of 'Takacs' equation' type. QUEUES WITH IMPATIENT UNITS AND REPEATED ATTEMPTS; TAKACS' EQUATION;

Asymptotic analysis of single resource loss systems in heavy traffic, with applications to integrated networks

Abstract: In this paper we consider the analysis of call blocking at a single resource with differing capacity requirements as well as differing arrival rates and holding times. We include in our analysis trunk reservation parameters which provide an important mechanism for tuning the relative call blockings to desired levels. We base our work on an asymptotic regime where the resource is in heavy traffic. We further derive, from our asymptotic analysis. methods for the analysis of finite systems. Empirical results suggest that these methods perform well for a wide class of examples.

A ratio limit theorem for (sub) markov chains on {1,2, * } with bounded jumps

Abstract: We consider positive matrices Q, indexed by {1,2, * }. Assume that there exists a constant 1 - L dr + L > dr >j for some r. If Q satisfies some additional uniform irreducibility and aperiodicity assumptions, then for s > 0, Q has at most one positive s-harmonic function and at most one s-invariant measure /L. We use this result to show that if Q is also substochastic, then it has the strong ratio limit property, that is

Statistics of the Boolean model: from the estimation of means to the estimation of distributions

Abstract: Non-parametric estimators of the distribution of the grain of the Boolean model are considered. The technique is based on the study of point processes of tangent points in different directions related to the Boolean model. Their second- and higher-order characteristics are used to estimate the mean body and the distribution of the typical grain. Central limit theorems for the improved estimator of the intensity and surface measures of the Boolean model are also proved

Mean size-and-shapes and mean shapes: a geometric point of view

Abstract: Unlike the means of distributions on a euclidean space, it is not entirely clear how one should define the means of distributions on the size-and-shape or shape spaces of k labelled points in Rn1, since these spaces are all curved. In this paper, we discuss, from a shape-theoretic point of view, some questions which arise in practice while using procrustean methods to define mean size-and-shapes or shapes. We obtain sufficient conditions for such means to be unique and for the corresponding generalized procrustean algorithms to converge to them. These conditions involve the curvature of the size-and-shape or shape spaces and are much less restrictive than asking for the data to be concentrated.

Transient characteristics of an M/M/∞ system

Abstract: Convergence results are given for transient characteristics of an M/M/∞ system such as the period of time the occupation process remains above a given state, the area swept by this process above this state and the number of customers arriving during this period. These results are precise in contrast to approximations derived in the framework of the Poisson clumping heuristic introduced by Aldous.

Limit laws for a stochastic process and random recursion arising in probabilistic modelling

Abstract: We study certain stochastic processes arising in probabilistic modelling. We discuss the limit behavior of these processes and estimate the rate of convergence to the limit

Solving the inverse problem for measures using iterated function systems: a new approach

Abstract: We present a systematic method of approximating, to an arbitrary accuracy, a probability measure A/ on x = [0, 1], q -1, with invariant measures for iterated function systems by matching its moments. There are two novel features in our treatment. 1. An infinite set of fixed affine contraction maps on X, W = {wl, w2, * * }, subject to an 'E-contractivity' condition, is employed. Thus, only an optimization over the associated probabilities pi is required. 2. We prove a collage theorem for moments which reduces the moment matching problem to that of minimizing the collage distance between moment vectors. The minimization procedure is a standard quadratic programming problem in the pi which can be solved in a finite number of steps. Some numerical calculations for the approximation of measures on [0,1] are presented.

The genealogy of branching processes and the age of our most recent common ancestor

Abstract: We obtain a weak approximation for the reduced family tree in a near-critical Markov branching process when the time interval considered is long ; we also extend Yaglom's theorem and the exponential law to this case. These results are then applied to the problem of estimating the age of our most recent common female ancestor, using mitochondrial DNA sequences taken from a sample of contemporary humans.

Birth and death processes as projections of higher-dimensional poisson processes

Abstract: Birth and death processes can be constructed as projections of higher-dimensional Poisson processes. The existence and uniqueness in the strong sense of the solutions of the time change problem are obtained. It is shown that the solution of the time change problem is equivalent to the solution of the corresponding martingale problem. Moreover, the processes obtained by the projection method are ergodic under translations.

A heuristic proof of a long-standing conjecture of d. g. kendall concerning the shapes of certain large random polygons

Abstract: In the early 1940s David Kendall conjectured that the shapes of the 'large' (i.e. large area A) convex polygons determined by a standard Poisson line process in the plane tend to circularity (as A increases). Subject only to one heuristic argument, this conjecture and the corresponding two results with A replaced in turn by number of sides N and perimeter S, are proved. Two further similar limiting distributions are considered and, finally, corresponding limiting non-deterministic shape distributions for the small polygons are determined.

QUASI-Stationary laws for Markov Processes: Examples of an always proximate absorbing state

Abstract: Under consideration is a continuous-time Markov process with non-negative integer state space and a single absorbing state 0. Let T be the hitting time of zero and suppose Pi(T t)=1 for all t>0. Most known cases satisfy (*). The Markov process has a quasi-stationary distribution iff Ei(eeT) 0. The published proof of this fact makes crucial use of (*). By means of examples it is shown that (*) can be violated in quite drastic ways without destroying the existence of a quasi-stationary distribution.

On almost optimal priority rules for preemptive scheduling of stochastic jobs on parallel machines

Abstract: We consider scheduling a batch of jobs with stochastic processing times on single or parallel machines, with the objective of minimizing the expected holding costs. Preemption of jobs is allowed, and the holding costs of preempted jobs may depend on the stage of completion. We provide a new proof of the optimality of a Gittins priority rule for the single machine and use the same proof to show that the Gittins priority rule is nearly optimal for parallel machines.

Generation of Johnson-Mehl crystals and comparative analysis of models for random nucleation

Abstract: Simulation procedures for typical Johnson-Mehl crystals generated under various models for random nucleation are proposed. These procedures include algorithms for simulating spatio-time-inhomogeneous Poisson processes. Empirical results for a particular class of Johnson-Mehl tessellations in two and three dimensions show remarkably different crystals.

The probability of large queue lengths and waiting times in a heterogeneous multiserver queue i: tight limits

Abstract: We consider a multiserver queuing process specified by i.i.d. interarrival time, batch size and service time sequences. In the case that different servers have different service time distributions we say the system is heterogeneous. In this paper we establish conditions for the queuing process to be characterized as a geometrically Harris recurrent Markov chain, and we characterize the stationary probabilities of large queue lengths and waiting times. The queue length is asymptotically geometric and the waiting time is asymptotically exponential. Our analysis is a generalization of the well-known characterization of the GI/G/1 queue obtained using classical

Markov-modulated traffic with nearly complete decomposability characteristics and associated fluid queueing models

Abstract: This paper considers fluid queuing models of Markov-modulated traffic that, due to large differences in the time-scales of events, possess structural characteristics that yield a nearly completely decomposable (NCD) state-space. Extension of domain decomposition and aggregation techniques that apply to approximating the eigensystem of Markov chains permits the approximate subdivision of the full system to a number of small, independent subsystems (decomposition phase), plus an ‘aggregative' system featuring a state-space that distinguishes only one index per subsystem (aggregation phase). Perturbation analysis reveals that the error incurred by the approximation is of an order of magnitude equal to the weak coupling of the NCD Markov chain. The study in this paper is then extended to the structure of NCD fluid models describing source superposition (multiplexing). It is shown that efficient spectral factorization techniques that arise from the Kronecker sum form of the global matrices can be applied through and combined with the decomposition and aggregation procedures. All structural characteristics and system parameters are expressible in terms of the individual sources multiplexed together, rendering the construction of the global system unnecessary. Finally, besides providing efficient computational algorithms, the work in this paper can be recast as a conceptual framework for the better understanding of queueing systems under the presence of events happening in widely differing time-scales.

The Estimation of mean shape and mean particle number in overlapping particle systems in the plane

Abstract: A stationary (but not necessarily isotropic) Boolean model Y in the plane is considered as a model for overlapping particle systems. The primary grain (i.e. the typical particle) is assumed to be simply connected, but no convexity assumptions are made. A new method is presented to estimate the intensity γ of the underlying Poisson process (i.e. the mean number of particles per unit area) from measurements on the union set Y. The method is based mainly on the concept of convexification of a non-convex set, it also produces an unbiased estimator for a (suitably defined) mean body of Y, which in turn makes it possible to estimate the mean grain of the particle process

Stereology for some classes of polyhedrons

Abstract: A general method for solving stereological problems for particle systems is applied to polyhedron structures. We suggested computing the kernel function of the respective stereological integral equation by means of computer simulation. Two models of random polyhedrons are investigated. First, regular prisms are considered which are described by their size and shape. The size-shape distribution of a stationary and isotropic spatial ensemble of regular prisms can be estimated from the size-shape distribution of the polygons observed in a section plane. Secondly, random polyhedrons are constructed as the convex hull of points which are uniformly distributed on surfaces of spheres. It is assumed that the size of the polyhedrons and the number of points (i.e. the number of vertices) are random variables. Then the distribution of a spatially distributed ensemble of polyhedrons is determined by its size-number distribution. The corresponding numerical density of this bivariate size-number distribution can be stereologically determined from the estimated numerical density of the bivariate size-number distribution of the intersection profiles.

Dependencies in markovian networks

Abstract: Monotonicity and correlation results for queueing network processes, generalized birth-death procgsses and generalized migration processes are obtained with respect to various orderings of the state space. We prove positive (e.g. association) and negative (e.g. negative association) correlations in space and positive correlations in time for different situations, in steady state as well as in the transient phase of the system. This yields exact bounds for joint probabilities in terms of their independent versions.

The probability of large queue lengths and waiting times in a heterogeneous multiserver queue II: Positive recurrence and logarithmic limits

Abstract: We continue our investigation of the batch arrival-heterogeneous multiserver queue begun in Part I. In a general setting we prove the positive Harris recurrence of the system, and with no additional conditions we prove logarithmic tail limits for the stationary queue length and waiting time distributions.

The rao-blackwell theorem in stereology and some counterexamples

Abstract: A version of the Rao–Blackwell theorem is shown to apply to most, but not all, stereological sampling designs. Estimators based on random test grids typically have larger variance than quadrat estimators; random s-dimensional samples are worse than random r-dimensional samples for s < r. Furthermore, the standard stereological ratio estimators of different dimensions are canonically related to each other by the Rao–Blackwell process. However, there are realistic cases where sampling with a lower-dimensional probe increases efficiency. For example, estimators based on (conditionally) non-randomised test point grids may have smaller variance than quadrat estimators. Relative efficiency is related to issues in geostatistics and the theory of wide-sense stationary random fields. A uniform minimum variance unbiased estimator typically does not exist in our context.

Limit theorems for the time of completion of Johnson-Mehl tessellations

Abstract: Johnson-Mehl tessellations can be considered as the results of spatial birth-growth processes. It is interesting to know when such a birth-growth process is completed within a bounded region. This paper deals with the limiting distributions of the time of completion for various models of Johnson-Mehl tessellations in R d and k-dimensional sectional tessellations, where 1 ≤ k < d, by considering asymptotic coverage probabilities of the corresponding Boolean models. Random fractals as the results of birth-growth processes are also discussed in order to show that a birth-growth process does not necessarily lead to a Johnson-Mehl tessellation.

Loss networks under diverse routing: the symmetric star network

Abstract: A highly symmetric loss network is considered, the symmetric star network previously considered by Whitt [12] and Ziedins and Kelly [14]. As the number of links, K, becomes large, the state space for this process also grows, so we consider a functional of the network, one which contains all information relevant to blocking probabilities within the network but which is easier to analyse. We show that this reduced process obeys a functional law of large numbers and a functional central limit theorem, the limit in this latter case being an Ornstein-Uhlenbeck diffusion process. Finally, by considering the network in equilibrium, we are able to prove that the well-known Erlang fixed point approximation for blocking probabilities is correct to within o(K −1/2 ) as K → ∞

Comparison of replacement policies via point processes

Abstract: First, some basic concepts from the theory of point processes are recalled and expanded. Then some notions of stochastic comparisons, which compare whole processes, are introduced. The use of these notions is illustrated by stochastically comparing renewal and related processes. Finally, applications of the different notions of stochastic ordering of point processes to many replacement policies are given.

Stereological estimation of particle size distributions

Abstract: The corpuscle problem of Wicksell is discussed. We give a numerical quadrature of Gauss–Chebyshev type for Wicksell's integral equation which combines a size distribution of discs on a sectional plane with that of spheres. We also give an estimation procedure of three-dimensional size distributions based on this quadrature and examine its theoretical properties. In practice, we need a smoothing technique for empirical distribution functions before applying this estimator. Simulation results are given. Our idea also is applied to the thick section case and an analysis of microscopic data is given.