Showing papers by "Ioannis Karatzas published in 1981"

The monotone follower problem in stochastic decision theory

Abstract: We consider the problem of optimally tracking the "random demand"x+wt, w. Brownian motion, by a nondecreasing processź. adapted to the Brownian past, so as to minimize the expected lossEź0Tź(x+wtźźt)dt. The decision problem is reduced to a free boundary one, and the latter is studied and solved for a large class of cost functionsź(ź).

Optimal stationary linear control of the Wiener process

Abstract: In the present paper, we consider the following stochastic control problem: to minimize the average expected total cost $$J(x,u) = \mathop {\lim \inf }\limits_{T \to \infty } (1/T)E_x^u \int_0^T {\left[ {\phi (\xi _t ) + |u_t (\xi )|} \right]} dt,$$ 〈subject to $$d\xi _t = u_1 (\xi )dt + dw_t , \xi _0 = x, |u| \leqslant 1,$$ (wt) a Wiener process, with all measurable functions on the past of the state process {ξs;s≤t} and bounded by unity, admissible as controls. It is proved that, under very mild conditions on the running cost function φ(·), the optimal law is of the form $$\begin{gathered} u_t^* (\xi ) = - sign\xi _t , |\xi _t | > b, \hfill \\ u_t^* (\xi ) = 0, |\xi _t | > b. \hfill \\ \end{gathered} $$ The cutoff pointb and the performance rate of the optimal lawu* are simultaneously determined in terms of the function φ(·) through a simple system of integrotranscendental equations.

Filtering for piecewise linear drift and observation

Abstract: The filtering problem for piecewise linear drift and observation functions is reduced to an initial-boundary value problem. The "corners" give rise to local time terms. A finite number of sufficient statistics appear, in the form of the values and one-sided derivatives of the conditional density at the "corners", or more generally in the form of weights in a representation of the conditional density by potentials. Both kinds of statistics propagate according to linear Volterra equations, and must be considered as infinite-dimensional. The theory developed here for piecewise linear dynamics enhances the study of the general nonlinear filtering problem in a natural way: Nonlinear functions can be approximated over bounded intervals by polygons, to any degree of accuracy; by constructing or calculating the optimal filter for the approximating piecewise linear dynamics as indicated in this paper, one can conceivably obtain very good sub-optimal filters for general nonlinear dynamics. That the results extend to many dimensions is far from clear, but likely whenever the necessary local times can be defined.

A Degree Method for Free Boundaries in Stochastic Control

Abstract: In stochastic control problems with a bounded control set, the Bellman-Hamilton Jacobi equation leads to two-sided free-boundary problems for the switching surfaces, expressible as an equivalent set of integral equations containing the boundary functions in a very implicit way that seems to preclude the standard method used in the Stefan problem. It is natural then to try to use the topological Leray-Schauder methods to study the properties of solutions. We apply such an approach to the sample problem:$\min _u E\int_0^T {[f(x_t ) + | {u(x_t ,t)} |]} dt$, subject to $dx_t = u(x_t ,t)dt + dw_t $, $| u | \leqq 1$, with $w_t $ a Wiener process. The absolute value cost $| u |$ leads to finding the boundaries of a “dead zone” in $(x,t)$-space that separates the zones $u = \pm 1$ for the optimal u . The a priori bounds requisite for the Leray-Schauder approach come from usual probabilistic and PDE estimates. Then the integral equations are shown to have the form (homeomorphism $ + $ compact) for which a degree t...

Filtering of diffusions controlled through their conditional measures

Abstract: For the closed-loop nonlinear filtering problem with control in separated form (a functional of the conditional distribution measure), the Kallianpur-Striebel formula becomes a stochastic equation for the unnormalized conditional distribution, given the past of the observations. Existence, uniqueness and measurability of solutions to this equation are discussed. The results give a partially positive answer to the question of admissibility of separated control laws. However, "pathological" nonanticipative but noncausal solutions appear here, after the manner of Tsirelson's example.