Showing papers in "Journal of Applied Probability in 1992"

Convergence theorems for a class of simulated annealing algorithms on ℝd

Abstract: We study a class of simulated annealing algorithms for global minimization of a continuous function defined on a subset of We consider the case where the selection Markov kernel is absolutely continuous and has a density which is uniformly bounded away from 0. This class includes certain simulated annealing algorithms recently introduced by various authors. We show that, under mild conditions, the sequence of states generated by these algorithms converges in probability to the global minimum of the function. Unlike most previous studies where the cooling schedule is deterministic, our cooling schedule is allowed to be adaptive. We also address the issue of almost sure convergence versus convergence in probability.

Analysis of the Luria–Delbrück distribution using discrete convolution powers

Abstract: The Luria–Delbruck distribution arises in birth-and-mutation processes in population genetics that have been systematically studied for the last fifty years. The central result reported in this paper is a new recursion relation for computing this distribution which supersedes all past results in simplicity and computational efficiency: p 0 = e–m ; where m is the expected number of mutations. A new relation for the asymptotic behavior of pn (≈ c/n 2) is also derived. This corresponds to the probability of finding a very large number of mutants. A formula for the z-transform of the distribution is also reported.

The first-passage density of the Brownian motion process to a curved boundary

Abstract: An expression for the first-passage density of Brownian motian to a curved boundary is expanded as a series of multiple integrals. Bounds are given for the error due to truncation of the series when the boundary is wholly concave or wholly convex. Extensions to the Brownian bridge and to continuous Gauss-Markov processes are given. The series provides a practical method for calculating the probability that a sample path crosses the boundary in a specified time-interval to a high degree of accuracy. A numerical example is given.

Useful martingales for stochastic storage processes with Lévy-type input

Abstract: We apply the general theory of stochastic integration to identify a martingale associated with a Levy process modified by the addition of a secondary process of bounded variation on every finite interval. This martingale can be applied to queues and related stochastic storage models driven by a Levy process. For example, we have applied this martingale to derive the (non-product-form) steady-state distribution of a two-node tandem storage network with Levy input and deterministic linear fluid flow out of the nodes.

The extremal index for a Markov chain

Abstract: The paper presents a method of computing the extremal index for a discrete-time stationary Markov chain in continuous state space. The method is based on the assumption that bivariate margins of the process are in the domain of attraction of a bivariate extreme value distribution. Scaling properties of bivariate extremes then lead to a random walk representation for the tail behaviour of the process, and hence to computation of the extremal index in terms of the fluctuation properties of that random walk. The result may then be used to determine the asymptotic distribution of extreme values from the Markov chain.

Sur certaines fonctionnelles exponentielles du mouvement brownien réel

Abstract: Dufresne [1] recently showed that the integral of the exponential of Brownian motion with negative drift is distributed as the reciprocal of a gamma variable. In this paper, it is shown that this result is another formulation of the distribution of last exit times for transient Bessel processes. A bivariate distribution of such integrals of exponentials is also obtained explicitly.

Non-homogeneous semi-Markov systems and maintainability of the state sizes

Abstract: In this paper we introduce and define for the first time the concept of a non-homogeneous semi-Markov system (NHSMS). The problem of finding the expected population stucture is studied and a method is provided in order to find it in closed analytic form with the basic parameters of the system. Moreover, the problem of the expected duration structure in the state is studied. It is also proved that all maintainable expected duration structures by recruitment control belong to a convex set the vertices of which are specified. Finally an illustration is provided of the present results in a manpower system. STOCHASTIC POPULATION MODELS; SEMI-MARKOV PROCESS; MANPOWER SYSTEMS; NON-HOMOGENEOUS MARKOV CHAINS AMS 1991 SUBJECT CLASSIFICATION: PRIMARY 90B70 SECONDARY 60J20

Stabilities and instabilities in population dynamics

Abstract: Stability in population size is illusory: populations left to themselves either grow beyond all bounds or die out. But if they do not die out their composition stabilizes. These problems are discussed in terms of general abstract, multitype branching processes. The life and descent of a typical individual is described.

Optimality of the Round Robin Routing Policy

Abstract: In this paper we consider the problem of routing customers to identical servers, each with its own infinite capacity queue. Under the assumptions that the service times form a sequence of independent and identically distributed random variables with increasing failure rate distribution and state information is not available, we establish that the round robin policy minimizes, in the sense of a separable increasing convex ordering, the customer response times and the numbers of customers in the queues. This work was supported in part by the National Science Foundation under grant ASC 88-8802764 and NCR-9116183. ******************************

The MacLaurin series for the GI/G/1 queue

Abstract: We derive the MacLaurin series for the moments of the system time and the delay with respect to the parameters in the service time or interarrival time distributions in the GI/G /1 queue. The coefficients in these series are expressed in terms of the derivatives of the interarrival time density function evaluated at zero and the moments of the service time distribution, which can be easily calculated through a simple recursive procedure. The light traffic derivatives can be obtained from these series. For the M/G /1 queue, we are able to recover the formulas for the moments of the system time and the delay, including the Pollaczek–Khinchin mean-value formula.

A boundary-crossing result for Brownian motion

Abstract: In this paper we obtain a new crossing result for Brownian motion. The boundary studied is a piecewise linear function consisting of two lines. The expression obtained for the boundary crossing probability is of a simple directly computable form.

Contact and chord length distributions of the Poisson Voronoi tessellation

Abstract: This paper presents the form of some characteristics of the Voronoi tessellation which is generated by a stationary Poisson process in . Expressions are given for the spherical and linear contact distribution functions. These formulae lead to numerically tractable double-integral formulae for chord length probability density functions.

On strict stationarity and ergodicity of a non-linear ARMA model

Abstract: Two recent papers by Petruccelli and Woolford (1984) and Chan et al. (1985) showed that the key element governing ergodicity of a threshold AR(1) model is the joint behavior of the two linear AR(1) pieces falling in the two boundary threshold regimes. They used essentially the necessary and sufficient conditions for ergodicity of a general Markov chain of Tweedie (1974), (1975) in a rather clever manner. However, it is difficult to extend the results to the more general threshold ARMA models. Besides, irreducibility is also required to apply Tweedie's results. In this paper, instead of pursuing the ideas in Tweedie's results, we shall develop a criterion similar in spirit to the technique used by Benes (1967) in the context of continuous-time Markov chains. Consequently, we derive a necessary and sufficient condition for existence of a strictly stationary solution of a general non-linear ARMA model to be introduced in Section 2 of this paper. This condition is then applied to the threshold ARMA(1, q) model to yield a sufficient condition for strict stationarity which is identical to the condition given by Petruccelli and Woolford (1984) for the threshold AR(1). Hence, the conjecture that the moving average component does not affect stationarity is partially verified. Furthermore, under an additional irreducibility assumption, ergodicity of a non-linear ARMA model is established. The paper then concludes with a necessary condition for stationarity of the threshold ARMA(1, q) model.

Work-modulated queues with applications to storage processes

Abstract: We study two FIFO single-server queueing models in which both the arrival and service processes are modulated by the amount of work in the system. In the first model, the nth customer's service time, Sn , depends upon their delay, Dn , in a general Markovian way and the arrival process is a non-stationary Poisson process (NSPP) modulated by work, that is, with an intensity that is a general deterministic function g of work in system V(t). Some examples are provided. In our second model, the arrivals once again form a work-modulated NSPP, but, each customer brings a job consisting of an amount of work to be processed that is i.i.d. and the service rate is a general deterministic function r of work. This model can be viewed as a storage (dam) model (Brockwell et al. (1982)), but, unlike previous related literature, (where the input is assumed work-independent and stationary), we allow a work-modulated NSPP. Our approach involves an elementary use of Foster's criterion (via Tweedie (1976)) and in addition to obtaining new results, we obtain new and simplified proofs of stability for some known models. Using further criteria of Tweedie, we establish sufficient conditions for the steady-state distribution of customer delay and sojourn time to have finite moments.

Rare events in series of queues

Abstract: In an ergodic network of K M/M/1 queues in series we consider the rare event that, as N increases, the total population in the network exceeds N during a busy period. By utilizing the contraction principle of large deviation theory, an action functional is obtained for this exit problem. The ensuing minimization is carried out for K = 2 and an indication is given for arbitrary K. It is shown that, asymptotically and for unequal service rates, the ‘most likely' path for this rare event is one where the arrival rate has been interchanged with the smallest service rate. The problem has been posed in Parekh and Walrand [7] in connection with importance sampling simulation methods for queueing networks. Its solution has previously been obtained only heuristically.

Optimality of index policies for stochastic scheduling with switching penalties

Abstract: We investigate the impact of switching penalties on the nature of optimal scheduling policies for systems of parallel queues without arrivals. We study two types of switching penalties incurred when switching between queues: lump sum costs and time delays. Under the assumption that the service periods of jobs in a given queue possess the same distribution, we derive an index rule that defines an optimal policy. For switching penalties that depend on the particular nodes involved in a switch, we show that although an index rule is not optimal in general, there is an exhaustive service policy that is optimal.

Estimation of the derivative of a stationary measure with respect to a control parameter

Abstract: The paper deals with a problem which arises in the Monte Carlo optimization of steady state or ergodic systems which can be modelled by Markov chains. The transition probability depends on a parameter, and one wishes to find the parameter value at which some performance function is minimum. The only available data are obtained from either simulation or actual operating information. For such a problem ore needs good statistical estimates of the derivatives. Conditions are given for the existence of the derivative of the stationary measure with respect to the parameter, in the sense that the derivative is a signed measure, and is the limit of the natural approximating sequence. Some properties and a useful characterization of the derivative are obtained. It is also shown that, under appropriate conditions, the derivative of the n-step transition function converges to the derivative of the stationary measure as n tends to oo. This latter result is of particular importance whether one is simply estimating or is actually optimizing via some sequential Monte Carlo procedure, since the basic observations are always taken over a finite time interval.

Time-dependent process of M/G/1 vacation models with exhaustive service

Abstract: Generalized M/G/1 vacation systems with exhaustive service include multiple and single vacation models and a setup time model possibly combined with an N-policy. In these models with given initial conditions, the time-dependent joint distribution of the server's state, the queue size, and the remaining vacation or service time is known (Takagi (1990)). In this paper, capitalizing on the above results, we obtain the Laplace transforms (with respect to time) for the distributions of the virtual waiting time, the unfinished work (backlog), and the depletion time. The steady-state limits of those transforms are also derived. An erroneous expression for the steady-state distribution of the depletion time in a multiple vacation model given by Keilson and Ramaswamy (1988) is corrected.

A closure characterisation of phase-type distributions

Abstract: We characterise the classes of continuous and discrete phase-type distributions in the following way. They are known to be closed under convolutions, mixtures, and the unary 'geometric mixture' operation. We show that the continuous class is the smallest family of distributions that is closed under these operations and contains all exponential distributions and the point mass at zero. An analogous result holds for the discrete class. We also show that discrete phase-type distributions can be regarded as R+-rational sequences, in the sense of automata theory. This allows us to view our characterisation of them as a corollary of the Kleene-Schiitzenberger theorem on the behavior of finite automata. We prove moreover that any summable R+-rational sequence is proportional to a discrete phase-type distribution. LAPLACE TRANSFORMS; GENERATING FUNCTIONS; AUTOMATA THEORY AMS 1991 SUBJECT CLASSIFICATION: PRIMARY 60J10 SECONDARY 68 Q75

Extremal properties of the FIFO discipline in queueing networks

Abstract: We show that using the FIFO service discipline at single server stations with ILR (increasing likelihood ratio) service time distributions in networks of monotone queues results in stochastically earlier departures throughout the network. The converse is true at stations with DLR (decreasing likelihood ratio) service time distributions. We use these results to establish the validity of the following comparisons: (i) The throughput of a closed network of FIFO single-server queues will be larger (smaller) when the service times are ILR (DLR) rather than exponential with the same means. (ii) The total stationary number of customers in an open network of FIFO single-server queues with Poisson external arrivals will be stochastically smaller (larger) when the service times are ILR (DLR) rather than exponential with the same means. We also give a surprising counterexample to show that although FIFO stochastically maximizes the number of departures by any time t from an isolated single-server queue with IHR (increasing hazard rate, which is weaker than ILR) service times, this is no longer true for networks of more than one queue. Thus the ILR assumption cannot be relaxed to IHR. Finally, we consider multiclass networks of exponential single-server queues, where the class of a customer at a particular station determines its service rate at that station, and show that serving the customer with the highest service rate (which is SEPT — shortest expected processing time first) results in stochastically earlier departures throughout the network, among all preemptive work-conserving policies. We also show that a cµ rule stochastically maximizes the number of non-defective service completions by any time t when there are random, agreeable, yields.

Zero-sum two-person semi-markov games

Abstract: Semi-Markov games are investigated under discounted and limiting average payoff criteria. The issue of the existence of the value and a pair of stationary optimal strategies are settled; the optimality equation is studied and under a natural ergodic condition the existence of a solution to the optimality equation is proved for the limiting average case. Semi-Markov games provide useful flexibility in constructing recursive game models. All the work on Markov/semi-Markov decision processes and Markov (stochastic) games can be viewed as special cases of the developments in this paper.

Some results on block replacement policies and renewal theory

Abstract: Age and block replacement policies are commonly used in order to reduce the number of in-service failures. The focus in this paper is on the block replacement policies, about which relatively less is known than age replacement policies. Several new results which connect the properties of block replacement policies with the properties of the corresponding renewal function and the excess lifetimes are obtained. Some applications and the relationships between these new results and some known results are included.

Steady-state Markov chain models for the Heine and Euler distributions

Abstract: The paper puts forward steady-state Markov chain models for the Heine and Euler distributions. The models for oil exploration strategies that were discussed by Benkherouf and Bather (1988) are reinterpreted as current-age models for discrete renewal processes. Steady-state success-runs processes with non-zero probabilities that a trial is abandoned, Foster processes, and equilibrium random walks corresponding to elective M / M /1 queues are also examined.

On the limiting distribution of a supercritical branching process in a random environment

Abstract: We consider an increasing supercritical branching process in a random environment and obtain bounds on the Laplace transform and distribution function of the limiting random variable. There are two possibilities that can be distinguished depending on the nature of the component distributions of the environment. If the minimum family size of each is 1, the growth will be as a power depending on a parameter a. If the minimum family sizes of some are greater than 1, it will be exponential, depending on a parameter y. We obtain bounds on the distribution function analogous to those found for the simple Galton-Watson case. It is not possible to obtain exact estimates and we are only able to obtain bounds to within e of the parameters.

Evaluation of the decay parameter for some specialized birth-death processes

Abstract: Let N(t) be an exponentially ergodic birth-death process on the state space {0, 1, 2, ···} governed by the parameters {λn, μn }, where µ 0 = 0, such that λn = λ and μn = μ for all n ≧ N, N ≧ 1, with λ < μ. In this paper, we develop an algorithm to determine the decay parameter of such a specialized exponentially ergodic birth-death process, based on van Doorn's representation (1987) of eigenvalues of sign-symmetric tridiagonal matrices. The decay parameter is important since it is indicative of the speed of convergence to ergodicity. Some comparability results for the decay parameters are given, followed by the discussion for the decay parameter of a birth-death process governed by the parameters such that lim n→∞ λn = λ and lim n→∞ µn = μ. The algorithm is also shown to be a useful tool to determine the quasi-stationary distribution, i.e. the limiting distribution conditioned to stay in {1, 2, ···}, of such specialized birth-death processes.

On equivalence of Markov properties over undirected graphs

Abstract: The dependence of coincidence of the global, local and pairwise Markov properties on the underlying undirected graph is examined. The pairs of these properties are found to be equivalent for graphs with some small excluded subgraphs. Probabilistic representations of the corresponding conditional independence structures are discussed.

Optimal block replacement policies with multiple choice at failure

Abstract: A generalization of the block replacement policy (BRP) is proposed and analysed. Under such a policy, an operating system is preventively replaced at times kT (k = 1, 2, 3, ···), independently of its failure history. At failure an operating system is either replaced by a new or a used one or minimally repaired or remains inactive until the next planned replacement. The cost of the ith minimal repair of the new subsystem at age y depends on the random part C(y) and the deterministic part ci (y). The mathematical model is defined and general analytical results are obtained.

On the optimality of LEPT and cµ rules for machines in parallel

Abstract: We consider scheduling problems with m machines in parallel and n jobs. The machines are subject to breakdown and repair. Jobs have exponentially distributed processing times and possibly random release dates. For cost functions that only depend on the set of uncompleted jobs at time t we provide necessary and sufficient conditions for the LEPT rule to minimize the expected cost at all t within the class of preemptive policies. This encompasses results that are known for makespan, and provides new results for the work remaining at time t. An application is that if the cµ rule has the same priority assignment as the LEPT rule then it minimizes the expected weighted number of jobs in the system for all t. Given appropriate conditions, we also show that the cµ rule minimizes the expected value of other objective functions, such as weighted sum of job completion times, weighted number of late jobs, or weighted sum of job tardinesses, when jobs have a common random due date.

Exploiting Markov chains to infer queue length from transactional data

Abstract: The use of taboo probabilities in Markov chains simplifies the task of calculating the queue-length distribution from data recording customer departure times and service commencement times such as might be available from automatic bank-teller machine transaction records or the output of telecommunication network nodes. For the case of Poisson arrivals, this permits the construction of a new simple exact O(n 3) algorithm for busy periods with n customers and an O(n 2 log n) algorithm which is empirically verified to be within any prespecified accuracy of the exact algorithm. The algorithm is extended to the case of Erlang-k interarrival times, and can also cope with finite buffers and the real-time estimates problem when the arrival rate is known.

On distributions having the almost-lack-of-memory property

Abstract: It is shown that random variables X exist, not exponentially or geometrically distributed, such that P(X - b =x IX b) = P(X 2 x) for all x > 0 and infinitely many different values of b. A class of distributions having the given property is exhibited. We call them ALM distributions, since they almost have the lack-of-memory property. For a given subclass of these distributions some phenomena relating to service by an unreliable server are discussed.