Topic
Bellman equation
About: Bellman equation is a research topic. Over the lifetime, 5884 publications have been published within this topic receiving 135589 citations.
Papers published on a yearly basis
Papers
More filters
TL;DR: In this paper, the authors give a necessary and sufficient condition for the existence of control that keeps the corresponding trajectory of the related stochastic control system within a prescribed closed subset of the state space.
Abstract: In this Note, we give a necessary and sufficient condition for the existence of control that keeps the corresponding trajectory of the related stochastic control system within a prescribed closed subset of the state space. The problem of existence of stochastic control under a state-constraint is also called the viability property of the underlying control system. Our result is: the square of the distance function of this constraint is a viscosity supersolution of a Hamilton-Jacobi-Bellman equation if and only if the system enjoys the viability property.
69 citations
TL;DR: This work considers a stochastic control problem that has emerged in the economics literature as an investment model under uncertainty and finds that this has a priori rather unexpected features.
Abstract: We consider a stochastic control problem that has emerged in the economics literature as an investment model under uncertainty This problem combines features of both stochastic impulse control and optimal stopping The aim is to discover the form of the optimal strategy It turns out that this has a priori rather unexpected features The results that we establish are of an explicit nature We also construct an example whose value function does not possess C1 regularity
69 citations
01 Jan 2009
TL;DR: It is proved that a non-dominated path should contain no cycles if random link travel times are consistent with the stochastic first-in-first-out principle, and it is shown that the optimal solution is a set of non- dominated paths under the first-order stochastically dominance.
Abstract: This paper studies the problem of finding most reliable a priori shortest paths (RASP) in a stochastic and time-dependent network. Correlations are modeled by assuming the probability density functions of link traversal times to be conditional on both the time of day and link states. Such correlations are spatially limited by the Markovian property of the link states, which may be such defined to reflect congestion levels or the intensity of random disruptions. We formulate the RASP problem with the above correlation structure as a general dynamic programming problem, and show that the optimal solution is a set of non-dominated paths under the first-order stochastic dominance. Conditions are proposed to regulate the transition probabilities of link states such that Bellman’s principle of optimality can be utilized. We prove that a non-dominated path should contain no cycles if random link travel times are consistent with the stochastic first-in-first-out principle. The RASP problem is solved using a non-deterministic polynomial label correcting algorithm. Approximation algorithms with polynomial complexity may be achieved when further assumptions are made to the correlation structure and to the applicability of dynamic programming. Numerical results are provided.
69 citations
TL;DR: In this article, a viscosity solution theory for a system of nonlinear degenerate parabolic integro-partial differential equations (IPDEs) related to stochastic optimal switching and control problems was developed.
Abstract: We develop a viscosity solution theory for a system of nonlinear degenerate parabolic integro-partial differential equations (IPDEs) related to stochastic optimal switching and control problems or stochastic games. In the case of stochastic optimal switching and control, we prove via dynamic programming methods that the value function is a viscosity solution of the IPDEs. In our setting the value functions or the solutions of the IPDEs are not smooth, so classical verification theorems do not apply.
69 citations
TL;DR: It is shown how a stochastic viability kernel and viable feedbacks relying on probability (or chance) constraints can be defined and computed by a dynamic programming equation.
69 citations