General irreducible Markov chains and non-negative operators: Preface

This paper considers discrete-time Markov control processes on Borel spaces, with possibly unbounded costs, and the long run average cost (AC) criterion. Under appropriate hypotheses on weighted norms for the cost function and the transition law, the existence of solutions to the average cost optimality inequality and the average cost optimality equation are shown, which in turn yield the existence of AC-optimal and AC-canonical policies respectively.

/pdf/average-cost-markov-control-processes-with-weighted-norms-3htbaxyqz8.pdf

Average cost Markov control processes with weighted norms: existence of canonical policies

Redundant data transmission in control/estimation over lossy networks

This paper considers constrained Markov control processes in Borel spaces, with unbounded costs. The criterion to be minimized is a long-run expected average cost, and the constraints can be imposed on similar average costs, or on average rewards, or discounted costs or rewards. We give conditions under which the constrained problem (CP) is solvable and equivalent to an equality constrained (EC) linear program. Furthermore, we show that there is no duality gap between EC and the dual program EC* and that in fact the strong duality condition holds. Finally, we introduce an explicit procedure to solve CP in some cases which is illustrated with a detailed example.

Constrained Average Cost Markov Control Processes in Borel Spaces

We are concerned with the expected average cost optimal control problem for discrete-time Markov control processes with Borel spaces and possibly unbounded costs. We show, under a Lyapunov stability condition and a growth condition on the costs, the existence of an stationary optimal policy using a well-known fixed point theorem. 2000 Mathematics Subject Classification: 90C40, 93E20.

The average cost optimality equation: A fixed point approach

This paper shows the convergence of the value iteration (or successive approximations) algorithm for average cost (AC) Markov control processes on Borel spaces, with possibly unbounded cost, under appropriate hypotheses on weighted norms for the cost function and the transition law. It is also shown that the aforementioned convergence implies strong forms of AC-optimality and the existence of forecast horizons.

E Gordienko

Papers

Average cost markov control processes with weighted norms : value iteration