# Discretisation of stochastic control problems for continuous time dynamics with delay

Abstract: As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretising time and space. A new feature in this context is to allow for delay in the dynamics. The existence of an optimal strategy with respect to the cost functional can be guaranteed in the class of relaxed controls. Weak convergence of the approximating extended Markov chains to the original process together with convergence of the associated optimal strategies is established.

...read more

##### Citations

77 citations

### Cites background from "Discretisation of stochastic contro..."

...More recent references on this approach for stochastic delay control problems are the papers [24, 25, 38]....

[...]

10 citations

### Cites methods from "Discretisation of stochastic contro..."

...The techniques used in [22] and related work for the proofs of convergence are based on weak convergence of measures; they can be extended to cover control problems with delay, see [11, 20]....

[...]

10 citations

### Cites methods from "Discretisation of stochastic contro..."

...problem reduces to a ﬁnite-dimensional one [2, 9, 28, 30]. In the general case, [27] extends the Markov chain approximation method to stochastic equations with delay. A similar method is developed in [32], and [17] establish convergence rates for an approximation of this kind. The inﬁnitedimensional Hilbertian approach to controlled deterministic and stochastic systems with delays in the state variabl...

[...]

5 citations

3 citations

##### References

5,022 citations

2,192 citations

### "Discretisation of stochastic contro..." refers background in this paper

...We consider control problems in the weak formulation (cf. Yong and Zhou, 1999: p. 64)....

[...]

...It then follows from a theorem by Weierstraÿ that Ĵ(ϕ, .) attains its minimum at some point of its compact domain (cf. Yong and Zhou, 1999: p. 65)....

[...]

^{1}

1,693 citations

1,136 citations

### "Discretisation of stochastic contro..." refers background in this paper

...For an exposition of the general theory of SDDEs see Mohammed (1984) or Mao (1997)....

[...]

624 citations

### "Discretisation of stochastic contro..." refers background in this paper

...For an exposition of the general theory of SDDEs see Mohammed (1984) or Mao (1997)....

[...]