Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

The author's preface gives an outline: "This book is about weakconvergence methods in metric spaces, with applications sufficient to show their power and utility. The Introduction motivates the definitions and indicates how the theory will yield solutions to problems arising outside it. Chapter 1 sets out the basic general theorems, which are then specialized in Chapter 2 to the space C[0, l ] of continuous functions on the unit interval and in Chapter 3 to the space D [0, 1 ] of functions with discontinuities of the first kind. The results of the first three chapters are used in Chapter 4 to derive a variety of limit theorems for dependent sequences of random variables. " The book develops and expands on Donsker's 1951 and 1952 papers on the invariance principle and empirical distributions. The basic random variables remain real-valued although, of course, measures on C[0, l ] and D[0, l ] are vitally used. Within this framework, there are various possibilities for a different and apparently better treatment of the material. More of the general theory of weak convergence of probabilities on separable metric spaces would be useful. Metrizability of the convergence is not brought up until late in the Appendix. The close relation of the Prokhorov metric and a metric for convergence in probability is (hence) not mentioned (see V. Strassen, Ann. Math. Statist. 36 (1965), 423-439; the reviewer, ibid. 39 (1968), 1563-1572). This relation would illuminate and organize such results as Theorems 4.1, 4.2 and 4.4 which give isolated, ad hoc connections between weak convergence of measures and nearness in probability. In the middle of p. 16, it should be noted that C*(S) consists of signed measures which need only be finitely additive if 5 is not compact. On p. 239, where the author twice speaks of separable subsets having nonmeasurable cardinal, he means "discrete" rather than "separable." Theorem 1.4 is Ulam's theorem that a Borel probability on a complete separable metric space is tight. Theorem 1 of Appendix 3 weakens completeness to topological completeness. After mentioning that probabilities on the rationals are tight, the author says it is an

Convergence of probability measures

This report presents a unified approach for the study of constrained Markov decision processes with a countable state space and unbounded costs. We consider a single controller having several objectives; it is desirable to design a controller that minimize one of cost objective, subject to inequality constraints on other cost objectives. The objectives that we study are both the expected average cost, as well as the expected total cost (of which the discounted cost is a special case). We provide two frameworks: the case were costs are bounded below, as well as the contracting framework. We characterize the set of achievable expected occupation measures as well as performance vectors. This allows us to reduce the original control dynamic problem into an infinite Linear Programming. We present a Lagrangian approach that enables us to obtain sensitivity analysis. In particular, we obtain asymptotical results for the constrained control problem: convergence of both the value and the policies in the time horizon and in the discount factor. Finally, we present and several state truncation algorithms that enable to approximate the solution of the original control problem via finite linear programs.

/pdf/constrained-markov-decision-processes-3ndxrfjbtt.pdf

Constrained Markov Decision Processes

“Multivalued Analysis” is the theory of set-valued maps (called multifonctions) and has important applications in many different areas. Multivalued analysis is a remarkable mixture of many different parts of mathematics such as point-set topology, measure theory and nonlinear functional analysis. It is also closely related to “Nonsmooth Analysis” (Chapter 5) and in fact one of the main motivations behind the development of the theory, was in order to provide necessary analytical tools for the study of problems in nonsmooth analysis. It is not a coincidence that the development of the two fields coincide chronologically and follow parallel paths. Today multivalued analysis is a mature mathematical field with its own methods, techniques and applications that range from social and economic sciences to biological sciences and engineering. There is no doubt that a modern treatise on “Nonlinear Functional Analysis” can not afford the luxury of ignoring multivalued analysis. The omission of the theory of multifunctions will drastically limit the possible applications.

SET-Valued Analysis

This chapter relates the notions of mutations with the concept of graphical derivatives of set-valued maps and more generally links the above results of morphological analysis with some basic facts of set-valued analysis that we shall recall.

Set-valued analysis

A technique different from the boundary layer method is developed to deal with singularly perturbed optimal control problems. The technique is applicable, in particular, in the case when the optima...

Suboptimization of singularly perturbed control systems

We investigate relationships between the deterministic infinite time horizon optimal control problem with discounting, in which the state trajectories remain in a given compact set $Y$, and a certain infinite dimensional linear programming (IDLP) problem. We introduce the problem dual with respect to this IDLP problem and obtain some duality results. We construct necessary and sufficient optimality conditions for the optimal control problem under consideration, and we give a characterization of the viability kernel of $Y$. We also indicate how one can use finite dimensional approximations of the IDLP problem and its dual for construction of near optimal feedback controls. The construction is illustrated with a numerical example.

Linear Programming Approach to Deterministic Infinite Horizon Optimal Control Problems with Discounting

The limit as ɛ→ 0 of the value function of a singularly perturbed optimal control problem is characterized. Under general conditions it is shown that limit value functions exist and solve in a viscosity sense a Hamilton—Jacobi equation. The Hamiltonian of this equation is generated by an infinite horizon optimization on the fast time scale. In particular, the limit Hamiltonian and the limit Hamilton—Jacobi equation are applicable in cases where the reduction of order, namely setting ɛ = 0 , does not yield an optimal behavior.

The value function of singularly perturbed control systems

Controlled coupled slow and fast motions are examined in a singular perturbations setting. The objective is to minimize a cost functional that takes into account both the fast motion, supposing, say, tracking a fast target, and the slow dynamics. A method is offered to cope with the possibility that the fast flow has nonstationary limits. Invariant measures of the fast motion are then the controlled objects on the infinitesimal scale. Optimal amalgamation of them on the slow scale induces the variational limit, whose solutions are near optimal solutions of the perturbed system.

Tracking Fast Trajectories Along a Slow Dynamics: A Singular Perturbations Approach

We establish that deterministic long run average problems of optimal control are "asymptotically equivalent" to infinite-dimensional linear programming problems (LPPs) and we establish that these LPPs can be approximated by finite-dimensional LPPs, the solutions of which can be used for construction of the optimal controls. General results are illustrated with numerical examples.

Vladimir Gaitsgory

Papers

Suboptimization of singularly perturbed control systems

Linear Programming Approach to Deterministic Infinite Horizon Optimal Control Problems with Discounting

The value function of singularly perturbed control systems

Tracking Fast Trajectories Along a Slow Dynamics: A Singular Perturbations Approach

Linear Programming Approach to Deterministic Long Run Average Problems of Optimal Control