The author's preface gives an outline: "This book is about weakconvergence methods in metric spaces, with applications sufficient to show their power and utility. The Introduction motivates the definitions and indicates how the theory will yield solutions to problems arising outside it. Chapter 1 sets out the basic general theorems, which are then specialized in Chapter 2 to the space C[0, l ] of continuous functions on the unit interval and in Chapter 3 to the space D [0, 1 ] of functions with discontinuities of the first kind. The results of the first three chapters are used in Chapter 4 to derive a variety of limit theorems for dependent sequences of random variables. " The book develops and expands on Donsker's 1951 and 1952 papers on the invariance principle and empirical distributions. The basic random variables remain real-valued although, of course, measures on C[0, l ] and D[0, l ] are vitally used. Within this framework, there are various possibilities for a different and apparently better treatment of the material. More of the general theory of weak convergence of probabilities on separable metric spaces would be useful. Metrizability of the convergence is not brought up until late in the Appendix. The close relation of the Prokhorov metric and a metric for convergence in probability is (hence) not mentioned (see V. Strassen, Ann. Math. Statist. 36 (1965), 423-439; the reviewer, ibid. 39 (1968), 1563-1572). This relation would illuminate and organize such results as Theorems 4.1, 4.2 and 4.4 which give isolated, ad hoc connections between weak convergence of measures and nearness in probability. In the middle of p. 16, it should be noted that C*(S) consists of signed measures which need only be finitely additive if 5 is not compact. On p. 239, where the author twice speaks of separable subsets having nonmeasurable cardinal, he means "discrete" rather than "separable." Theorem 1.4 is Ulam's theorem that a Borel probability on a complete separable metric space is tight. Theorem 1 of Appendix 3 weakens completeness to topological completeness. After mentioning that probabilities on the rationals are tight, the author says it is an

Convergence of probability measures

Matrix Differential Calculus with Applications in Statistics and Econometrics

Advanced Mathematical Methods for Scientists and Engineers

A first course in stochastic processes

In this note, we consider the problems of stochastic stability and sliding-mode control for a class of linear continuous-time systems with stochastic jumps, in which the jumping parameters are modeled as a continuous-time, discrete-state homogeneous Markov process with right continuous trajectories taking values in a finite set. By using Linear matrix inequalities (LMIs) approach, sufficient conditions are proposed to guarantee the stochastic stability of the underlying system. Then, a reaching motion controller is designed such that the resulting closed-loop system can be driven onto the desired sliding surface in a limited time. It has been shown that the sliding mode control problem for the Markovian jump systems is solvable if a set of coupled LMIs have solutions. A numerical example is given to show the potential of the proposed techniques.

https://www.researchgate.net/profile/Peng_Shi5/publication/3032315_On_designing_of_sliding-mode_control_for_stochastic_jump_systems/links/0c960519604c828e4a000000.pdf

On designing of sliding-mode control for stochastic jump systems

In this paper the authors consider a preventive maintenance and production model of a flexible manufacturing system with machines that are subject to breakdown and repair. The preventive maintenance can be used to reduce the machine failure rates and improve the productivity of the system. The control variables are the rate of maintenance and the rate of production; the objective is to choose a control process that optimizes a robust cost of inventory/shortage, production, and maintenance. The value function is shown to be locally Lipschitz and to satisfy a Hamilton-Jacobi-Isaacs equation. A sufficient condition for optimal control is obtained. Finally, an algorithm is given for solving the optimal control problem numerically. >

Robust production and maintenance planning in stochastic manufacturing systems

The paper is concerned with hybrid control of a class of linear quadratic Gaussian (LQG) systems modulated by a finite-state Markov chain. It develops approximation schemes for systems involving singularly perturbed Markov chains with weak and strong interactions, which are useful for natural time-scale separation and for large-scale Markovian systems. The presentation encompasses three cases: the recurrent Markov chains, inclusion of transient states, and inclusion of absorbing states. Near optimality of the approximation schemes is obtained, and a numerical example is presented. Computational results indicate our approximation schemes perform well.

On nearly optimal controls of hybrid LQG problems

Originating from a wide range of applications in optimization and control of large-scale systems (such as telecommunications, queueing networks, and manufacturing systems), this work is devoted to a class of singularly perturbed discrete-time Markov chains. The states of the Markov chain are naturally decomposable into recurrent and transient classes such that within each class the interactions are strong and among different classes the interactions are weak. Our study focuses on the difference equations representing the probabilityvector, and aims at deriving matched asymptotic expansions of the solutions. Justification and error analysis are also provided. Both time-homogeneous and time-inhomogeneous models are examined.

Singularly Perturbed Discrete-Time Markov Chains

Input design is of essential importance in system identification for providing sufficient probing capabilities to guarantee convergence of parameter estimates to their true values. This paper presents conditions on input signals that characterize their probing richness for strongly consistent parameter estimation of linear systems with binary-valued output observations. Necessary and sufficient conditions on periodic signals are derived for sufficient richness. These conditions are further studied under different system configurations including open-loop and feedback systems, and different scenarios of noises including actuator noise, input measurement noise, and output measurement noise. In addition to system parameter estimation, essential properties of identifiability and input conditions are also derived when sensor thresholds or noise distribution functions are unknown. The findings of this paper provide a foundation to study identification of systems that either use binary-valued or quantized sensors or involve communication channels, which mandate quantization of signals.

/pdf/identification-input-design-for-consistent-parameter-485v62hkyq.pdf

Identification Input Design for Consistent Parameter Estimation of Linear Systems With Binary-Valued Output Observations

We analyze the tracking performance of the least mean square (LMS) algorithm for adaptively estimating a time varying parameter that evolves according to a finite state Markov chain. We assume the Markov chain jumps infrequently between the finite states at the same rate of change as the LMS algorithm. We derive mean square estimation error bounds for the tracking error of the LMS algorithm using perturbed Lyapunov function methods. Then combining results in two-time-scale Markov chains with weak convergence methods for stochastic approximation, we derive the limit dynamics satisfied by continuous-time interpolation of the estimates. Unlike most previous analyzes of stochastic approximation algorithms, the limit we obtain is a system of ordinary differential equations with regime switching controlled by a continuous-time Markov chain. Next, to analyze the rate of convergence, we take a continuous-time interpolation of a scaled sequence of the error sequence and derive its diffusion limit. Somewhat remarkably, for correlated regression vectors we obtain a jump Markov diffusion. Finally, two novel examples of the analysis are given for state estimation of hidden Markov models (HMMs) and adaptive interference suppression in wireless code division multiple access (CDMA) networks.

/pdf/least-mean-square-algorithms-with-markov-regime-switching-t8rop2o29h.pdf

G.G. Yin

Papers

Robust production and maintenance planning in stochastic manufacturing systems

On nearly optimal controls of hybrid LQG problems

Singularly Perturbed Discrete-Time Markov Chains

Identification Input Design for Consistent Parameter Estimation of Linear Systems With Binary-Valued Output Observations

Least mean square algorithms with Markov regime-switching limit