Showing papers on "Hidden Markov model published in 1976"

Note on algorithm 21

Abstract: In algorithm 21 Spremann and Gessner [1] present a new algorithm for an ergodic Markov decision process. This note shows that this algorithm not necessarily converges and suggest a modified algorithm.

Estimation of feature-dependent markov process transition probability matrices

Abstract: This article is concerned with the estimation of Markov process transition probabilities for nonhomogeneous populations. This estimation problem is shown to be solvable using general pattern recognition techniques. Numerous multivariate estimation techniques exist in the field of statistical pattern recognition, and many of these will be useful to researchers who use Markov process models of a population's behavior. These techniques are particularly called for when the behavior of members of a population is suspected to depend upon a set of descriptive feature values. One such technique, that of linear discriminant analysis, is presented in detail as an illustration of a statistical pattern recognition approach to a specific Markov process estimation problem. A brief example is given.

Formulation and solution of a class of total technology problems using Markov models and modular programming

Abstract: By relating basic aspects of Markov theory to engineering systems, two results were achieved: (i) An original classification of problems that can be solved using Markov methods has been made, by defining the category of systems that can be modelled by a Markov process. It can be used to determine whether any given problem can be solved by Markov methods. In addition it identifies a large number of problems in various disciplines that are susceptible to these methods. (ii) A procedure outlining how to apply Markov methods is given. This is original as it is a GENERAL procedure for ANY problem in the Markov category. It includes the definition of sub-classes of system that can be represented by basic Markov equations; and the identification of a range of existing techniques for solving them. The procedure was expanded in more detail for Reliability problems: it defines the best, model to use; proves the relationship between the Markov parameters and the reliability data needed to formulate the model; and specifies how to compute some reliability parameters from Markov solutions for different systems. All these contributions listed are original. A suite of programs, called Polymark, have been developed to aid the solution of problems using Markov methods. The programs are novel in that they are application independent. Many diverse industrial systems can be modelled by the same Markov equation. Polymark solves a number of Markov equations and so can be used for many different problems. It is envisaged that Polymark will form the nucleus of a larger system, with added application orientated software packages to interface with the user. The programs were designed in a modular form, as this gives several advantages over conventional programs. The design strategy used incorporates some original points and so is included in the thesis.