Showing papers in "Journal of Applied Probability in 1990"

On an index policy for restless bandits

Abstract: We investigate the optimal allocation of effort to a collection of n projects. The projects are 'restless' in that the state of a project evolves in time, whether or not it is allocated effort. The evolution of the state of each project follows a Markov rule, but transitions and rewards depend on whether or not the project receives effort. The objective is to maximize the expected time-average reward under a constraint that exactly m of the n projects receive effort at any one time. We show that as m and n tend to oo with m/n fixed, the per-project reward of the optimal policy is asymptotically the same as that achieved by a policy which operates under the relaxed constraint that an average of m projects be active. The relaxed constraint was considered by Whittle (1988) who described how to use a Lagrangian multiplier approach to assign indices to the projects. He conjectured that the policy of allocating effort to the m projects of greatest index is asymptotically optimal as m and n tend to oo. We show that the conjecture is true if the differential equation describing the fluid approximation to the index policy has a globally stable equilibrium point. This need not be the case, and we present an example for which the index policy is not asymptotically optimal. However, numerical work suggests that such counterexamples are extremely rare and that the size of the suboptimality

An integer-valued pth-order autoregressive structure (INAR(p)) process

Abstract: An extension of the INAR(1) process which is useful for modelling discrete-time dependent counting processes is considered. The model investigated here has a form similar to that of the Gaussian AR(p) process, and is called the integer-valued pthorder autoregressive structure (INAR(p)) process. Despite the similarity in form, the two processes differ in many aspects such as the behaviour of the correlation, Markovian property and regression. Among other aspects of the INAR( p) process investigated here are the limiting as well as the joint distributions of the process. Also, some detailed discussion is given for the case in which the marginal distribution of the process is Poisson.

The rendezvous problem on discrete locations

Abstract: Two friends have become separated in a building or shopping mall and and wish to meet as quickly as possible. There are n possible locations where they might meet. However, the locations are identical and there has been no prior agreement where to meet or how to search. Hence they must use identical strategies and must treat all locations in a symmetrical fashion. Suppose their search proceeds in discrete time. Since they wish to avoid the possibility of never meeting, they will wish to use some randomizing strategy. If each person searches one of the n locations at random at each step, then rendezvous will require n steps on average. It is possible to do better than this: although the optimal strategy is difficult to characterize for general n, there is a strategy with an expected time until rendezvous of less than 0.829 n for large enough n. For n = 2 and 3 the optimal strategy can be established and on average 2 and 8/3 steps are required respectively. There are many tantalizing variations on this problem, which we discuss with some conjectures. DYNAMIC PROGRAMMING; SEARCH PROBLEMS

A paradox of congestion in a queuing network

Abstract: In an uncongested transportation network, adding routes and capacity to an existing network must decrease, or at worst not change, the average time individuals require to travel through the network from a source to a destination. Braess (1968) discovered that the same is not true in congested networks. Here we give an example of a queuing network in which added capacity leads to an increase in the mean transit time for everyone. Self-seeking individuals are unable to refrain from using the additional capacity, even though using it leads to deterioration in the mean transit time. This example appears to be the first queuing network to demonstrate the general principle that in non-co-operative games with smooth payoff functions, user-determined equilibria generically deviate from system-optimal equilibria.

The job search problem as an employer-candidate game

Abstract: In the standard job search problem a single decision-maker (say an employer) has to choose from a sequence of candidates of varying fitness. We extend this formulation to allow both employers and candidates to make choices. We consider an infinite population of employers and an infinite population of candidates. Each employer interviews a (possibly infinite) sequence of candidates for a post and has the choice of whether or not to offer a candidate the post. Each candidate is interviewed by a (possibly infinite) sequence of employers and can accept or reject each offer. Each employer seeks to maximise the fitness of the candidate appointed and each candidate seeks to maximise the fitness of their eventual employer. We allow both discounting and/or a cost per interview. We find that there is a unique pair of policies (for employers and candidates respectively) which is in Nash equilibrium. Under these policies each population is partitioned into a finite or countable sequence of subpopulations, such that an employer (candidate) in a given subpopulation ends up matched with the first candidate (employer) encountered from the corresponding subpopulation. In some cases the number of non-empty subpopulations in the two populations will differ and some members of one population will never be matched.

Monte Carlo simulation and large deviations theory for uniformly recurrent Markov chains

Abstract: We consider the estimation via simulation following the importance sampling technique of certain large deviations probabilities for time-homogeneous Markov chains. We first demonstrate that when the simulation distribution is also a homogeneous Markov chain, the estimator variance will vanish exponentially as the sample size n tends to ∞. We then prove that the estimator variance is asymptotically minimized by the same exponentially twisted Markov chain which arises in large deviation theory. This optimization is unique among uniformly recurrent homogeneous Markov chain simulation distributions

Equilibrium distribution of block-structured Markov chains with repeating rows

Abstract: In this paper we consider two-dimensional Markov chains with the property that, except for some boundary conditions, when the transition matrix is written in block form, the rows are identical except for a shift to the right. We provide a general theory for finding the equilibrium distribution for this class of chains. We illustrate theory by showing how our results unify the analysis of the M/G/1 and GI/M/1 paradigms introduced by M. F. Neuts.

On the GIX/G/∞ system

Abstract: By using a shot noise process, general results on system size in continuous time are given both in transient state and in steady state with discussion on some interesting results concerning special cases. System size before arrivals is also discussed.

On extremal service disciplines in single-stage queueing systems

Abstract: It is shown that among all work-conserving service disciplines that are independent of the future history, the first-come-first-served (FCFS) service discipline minimizes [maximizes] the average sojourn time in a G/GI/ 1 queueing system with new better [worse] than used in expectation (NBUE[NWUE]) service time distribution. We prove this result using a new basic identity of G/GI/ 1 queues that may be of independent interest. Using a relationship between the workload and the number of customers in the system with different lengths of attained service it is shown that the average sojourn time is minimized [maximized] by the least-attained-service time (LAST) service discipline when the service time has the decreasing [increasing] mean residual life (DMRL[IMRL]) property.

A Functional Central Limit Theorem for the Ewens Sampling Formula

Abstract: For each n > 0, the Ewens sampling formula from population genetics is a measure on the set of all partitions of the integer n. To determine the limiting distributions for the part sizes of a partition with respect to the measures given by this formula, we associate to each partition a step function on [0, 1]. Each jump in the function equals the number of parts in the partition of a certain size. We normalize these functions and show that the induced measures on D[0, 1] converge to Wiener measure. This result complements Kingman's frequency limit theorem [10] for the Ewens partition structure.

First-order autoregressive models for gamma and exponential processes

Abstract: In this paper we propose an autoregressive representation for a particular type of stationary Gamma(θ –1, v) process whose n-dimensional joint distributions have Laplace transform |In + θSnVn | –v , where Sn = diag(s 1, · ··, sn ), Vn is an n × n positive definite matrix with elements υ ij = p|i–j|i 2, i, j = 1, ···, n and p is the lag-1 autocorrelation of the gamma process. We also generalize the two-parameter NEAR(1) model of Lawrance and Lewis (1981) to an exponential first-order autoregressive model with three parameters. The correlation structure and higher-order properties of the two proposed models are also given.

On a daley-kendall model of random rumours

Abstract: Suppose that a certain population consists of N individuals. One member initially learns a rumour from an outside source, and starts telling it to other members, who continue spreading the information. A knower becomes inactive once he encounters somebody already informed. Daley and Kendall, who initiated the study of this model, conjectured that the number of eventual knowers is asymptotically normal with mean and variance linear in N. Our purpose is to confirm this conjecture.

Stationary distributions for piecewise-deterministic Markov processes

Abstract: In this paper we show that the problem of existence and uniqueness of stationary distributions for piecewise-deterministic Markov processes (PDPs) is equivalent to the same problem for the associated Markov chain, so long as some mild conditions on the parameters of the PDP are satisfied. Our main result is the construction of an invertible mapping from the set of stationary distributions for the PDP to the set of stationary distributions for the Markov chain. Some sufficient conditions for existence are presented and an application to capacity expansion is given.

On the two-boundary first-crossing-time problem for diffusion processes

Abstract: The first-crossing-time problem through two time-dependent boundaries for one-dimensional diffusion processes is considered. The first-crossing p.d.f.'s from below and from above are proved to satisfy a new system of Volterra integral equations of the second kind involving two arbitrary continuous functions. By a suitable choice of such functions a system of continuous-kernel integral equations is obtained and an efficient algorithm for its solution is provided. Finally, conditions on the drift and infinitesimal variance of the diffusion process are given such that the system of integral equations reduces to a non-singular single integral equation for the first-crossing-time p.d.f.

Laplace Transform Inversion and Passage-Time Distributions in Markov Processes

Abstract: Products of the Laplace transforms of exponential distributions with different parameters are inverted to give a mixture of Erlang densities, i.e. an expression for the convolution of exponentials. The formula for these inversions is expressed both as an explicit sum and in terms of a recurrence relation which is better suited to numerical computation. The recurrence for the inversion of certain weighted sums of these transforms is then solved by converting it into a linear first-order partial differential equation. The result may be used to find the density function of passage times between states in a Markov process and it is applied to derive an expression for cycle time distribution in tree-structured Markovian queueing networks.

On arrivals that see time averages: a martingale approach

Abstract: This paper is a sequel to our previous paper investigating when arrivals see time averages (ASTA) in a stochastic model; i.e., when the steady-state distribution of an embedded sequence, obtained by observing a continuous-time stochastic process just prior to the points (arrivals) of an associated point process, coincides with the steady-state distribution of the observed process. The relation between the two distributions was also characterized when ASTA does not hold. These results were obtained using the conditional intensity of the point process given the present state of the observed process (assumed to be well defined) and basic properties of Riemann–Stieltjes integrals. Here similar results are obtained using the stochastic intensity associated with the martingale theory of point processes, as in Bremaud (1981). In the martingale framework, the ASTA result is almost an immediate consequence of the definition of a stochastic intensity. In a stationary framework, the results characterize the Palm distribution, but stationarity is not assumed here. Watanabe's (1964) martingale characterization of a Poisson process is also applied to establish a general version of anti–PASTA: if the points of the point process are appropriately generated by the observed process and the observed process is Markov with left-continuous sample paths, then ASTA implies that the point process must be Poisson.

Empirical Laplace transform and approximation of compound distributions

Abstract: Let (Xn) be a sequence of non-negative random variables with distribution function F and Laplace transform L, and let N be an integer independent of the sequence. In many applications one knows that for y→∞ and a function φ P{Σ i=1 N X i >y}∼φ(y,τ,L(τ),L'(τ),...) where in turn τ is the solution of an equation ψ(τ,L(τ),...)=0. On the basis of a sample of size n we derive an estimator τ n for τ by solving ψ(τ n ,L n (τ n ),L' n (τ n ),...)=0 where L n is the empirical version of L. This estimator is then used to derive the asymptotic behaviour of φ(y,τ n ,L n (τ n ),L' n (τ n ),...). We include examples from insurance mathematics

Control of asymptotic variability in non-homogeneous Markov systems

Abstract: In this paper we provide two basic results. First, we find the set of all the limiting vectors of expectations, variances and covariances in an NHMS which are possible provided that we control the limit vector of the sequence of vectors of input probabilities. Secondly, under certain conditions easily met in practice we find the distribution of the limiting vector of expectations, variances and covariances to be multinomial with probabilities the corresponding limiting expected populations in the various states of the NHMS.

Complementary generating functions for the Mx/GI/1/k and GI/MY/1/k queues and their application to the comparison of loss probabilities

Abstract: A direct proof is presented for the fact that the stationary system queue length distribution just after the service completion epochs in the M x /GI/ 1 /k queue is given by the truncation of a measure on Z+ = {0, 1, ·· ·}. The related truncation formulas are well known for the case of the traffic intensity ρ M/GI/ 1 with a limited waiting time (Cohen (1982) and Takacs (1974)). By the duality of GI/M Y / 1 /k to M x /GI/ 1/ k + 1, we get a similar result for the system queue length distribution just before the arrival of a customer in GI/M Y / 1 /k. We apply those results to prove that the loss probabilities of M x /GI/ 1 /k and GI/M Y / 1 /k are increasing for the convex order of the service time and interarrival time distributions, respectively, if their means are fixed.

Certain state-dependent processes for dichotomised parasite populations

Abstract: The paper re-examines Quinn and MacGillivray's (1986) stationary birth-death process for a population of fixed size N consisting of two types of parasite, active and passive, and sets up a more elaborate model for the dichotomy between parasites on hosts with and without open wounds resulting from previous parasite attacks. The probability generating functions for the stationary count distributions are obtained, allowing limiting forms of the distributions to be investigated.

A unification of some approaches to Poisson approximation

Abstract: Let Sn be a sum of independent random variables. For the approximation of Sn by a Poisson random variable Y with the same mean, the complex analysis approaches based on generating functions and the semigroup approach are presented in a unified setting which permits us to refine Kerstan's complex analysis approach obtaining considerably sharper upper bounds for some metric distances of Sn and Y. These results are applied to some special Sn counting the records of an i.i.d. sequence of random variables which is important to various applied problems, for instance the secretary problem.

Characterization of the exponential distribution by the relevation transform

Abstract: A characterization of the exponential distribution was obtained by Grosswald et al. (1980) using the relevation transform introduced by Krakowski (1973). Here we obtain an improved version of the result in Grosswald et al. (1980).

Families of life distributions characterized by two moments

Abstract: We consider several classical notions of partial orderings among life distributions which have been used to describe ageing properties and tail domination. We show that if a distribution G dominates another distribution F in one of these partial orderings introduced here, and if two moments of G agree with those of F, including the moment that describes this partial ordering, then G = F. This leads to a characterization of the exponential distribution among HNBUE and HNWUE life distribution classes, and thus extends the results of Basu and Bhattacharjee (1984) and rectifies an error in that paper.

A spatial Markov property for nearest-neighbour Markov point processes

Abstract: Nearest-neighbour Markov point processes were introduced by Baddeley and Moller as generalizations of the Markov point processes of Ripley and Kelly. This note formulates and discusses a spatial Markov property for these point processes

On the Maximum and Its Uniqueness for Geometric Random Samples

Abstract: Given n independent, identically distributed random variables, let pn denote the probability that the maximum is unique. This probability is clearly unity if the distribution of the random variables is continuous. We explore the asymptotic behavior of the pn's in the case of geometric random variables. We find a function Q such that (pn - D(n)) -- 0 as n - 0o. In particular, we show that pn does not converge as n -- oo. We derive a related asymptotic result for the expected value of the maximum of the sample. These results arose out of a random depletion model due to

On information-based minimal repair and the reduction in remaining system lifetime due to the failure of a specific module

Abstract: The first part of this paper is inspired by a somewhat surprising result in Arjas and Norros (1989). Here we give some results comparing remaining system lifetime just after a ‘black box' minimal repair of a system and after a natural minimal repair based on information on the component level. In the second part we consider the reduction in remaining system lifetime due to the failure of a specific module and explore the relation to the reduction in remaining system lifetime due to the failure of a component inside the module. This former reduction also equals the increase in remaining system lifetime due to a minimal repair of the module at its time of failure. The expected value of this reduction/increase is proportional to the so-called Natvig measure of the importance of the module.

A convergence rate in extreme-value theory

Abstract: A uniform convergence rate is determined for maxima of i.i.d. random variables from a distribution in the domain of attraction of the double-exponential distribution. The result is proved under a second-order condition on the underlying distribution parallelling the one given in Smith (1982) for the domain of attraction of the bounded-below and bounded-above families of limit distributions.

Optimal R&D programs in a random environment

Abstract: Our study examines a stochastic R&D model with flexible termination time and without rivalry. Specifically, we assume a stochastic relationship between expenditures rate and the project's status.Furthermore, the termination time of the project is incorporated into the R&D model as a decision variable by allowing the controller to 'sell' the obtained technology from the project at any point of time. The proposed framework extends the classical approach in the R&D literature. The main purpose of our study is to determine the optimal stopping time of the project and to characterize qualitatively the firm's expenditure strategy. We show that under certain realistic conditions, the optimal stopping strategy is a control limit policy. Furthermore, the research effort increases monotonically over the development time of the project.

Poisson approximation for some epidemic models

Abstract: The Daniels' Poisson limit theorem for the final size of a severe general stochastic epidemic is extended to the Martin-Lof epidemic, and an order of magnitude for the error in the approximation is also given. The argument consists largely of showing that the number of survivors of a severe epidemic is essentially the same as the number of isolated vertices in a random directed graph. Poisson approximation for the latter quantity is proved using the Stein–Chen method and a suitable coupling.