A new algorithm for the hierarchical aggregation of singularly perturbed finite-state Markov processes is derived. The approach taken bridges the gap between conceptually simple results for a relatively restricted class of processes and the significantly more complex results for the general case. The critical role played by (almost) transient states is exposed, resulting in a straightforward algorithm for the construction of a sequence of aggregate generators associated with various time scales. These generators together provide a uniform asymptotic approximation of the original probability transition function.

The reduction of perturbed Markov generators: an algorithm exposing the role of transient states

ABSTRACf. This paper studies the aggregation/disaggregation of nearly completely decomposable Markov chains that have many applications in queueing networks and packet switched networks. A general class of simi­ larity transformation that transforms the stochastic transition probability matrix into a reduced order aggregated matrix is presented. This transfor­ mation is used to develop an aggregation algorithm to compute the exact stationary probability distribution, as weB as O( e k ) approximation of it. The proposed aggregation method is applied to a multiprogramming computer system with six active terminals and the capacity of the CPU and the secon­ dary memory is 3. This example is used to compare our algorithm with three well-known algorithms. The simulation studies showed that our algorithm usually converges in less number of iterations and CPU time. Moreover, it is shown that the other algorithms do not converge in some cases while our algorithm usually converges.

/pdf/aggregation-disaggregation-methods-for-computing-the-3f6zizk7px.pdf

Aggregation/Disaggregation Methods for Computing the Stationary Distribution of Markov Chains with Application to Multiprogramming System

A new algorithm for the hierarchical aggregation of singularly perturbed finite-state Markov processes is derived. The approach taken bridges the gap between conceptually simple results for a relatively restricted class of processes and the significantly more complex results for the general case. The critical role played by (almost) transient states is exposed resulting in a straightforward algorithm for the construction of a sequence of aggregate generators associated with various time scales. These generators together provide a uniform asymptotic approximation of the original probability transition function.

/pdf/an-algorithm-exposing-the-role-of-transient-states-skn81ul448.pdf

An Algorithm Exposing the Role of Transient States

: This paper reports on recent work on time scale decomposition and aggregation of large-scale linear systems containing weak couplings and finite-state Markov processes containing rare transitions. This work builds on that of Coderch, et. al.. The focus of the work is on the asymptotic approximatin of the linear system. (Author)

J.R. Rohlicek

Papers

Time Scale Decomposition: The Role of Scaling in Linear Systems and Transient States in Finite-State Markov Processes