Topic
Bellman equation
About: Bellman equation is a research topic. Over the lifetime, 5884 publications have been published within this topic receiving 135589 citations.
Papers published on a yearly basis
Papers
More filters
••
TL;DR: Monotonicity results for optimal policies of various queueing and resource sharing models are studied by concentrating on the events and the form of the value function instead of on thevalue function itself.
Abstract: In this paper we study monotonicity results for optimal policies of various queueing and resource sharing models. The standard approach is to propagate, for each specific model, certain properties of the dynamic programming value function. We propose a unified treatment of these models by concentrating on the events and the form of the value function instead of on the value function itself. This is illustrated with the systematic treatment of one and two-dimensional models.
169 citations
••
TL;DR: In this article, a general method for constructing high-order approximation schemes for Hamilton-Jacobi-Bellman equations is given, based on a discrete version of the Dynamic Programming Principle.
Abstract: A general method for constructing high-order approximation schemes for
Hamilton-Jacobi-Bellman equations is given. The method is based on a
discrete version of the Dynamic Programming Principle. We prove a
general convergence result for this class of approximation schemes also
obtaining, under more restrictive assumptions, an estimate in
$L^\infty$
of the order of convergence and of the local truncation error. The
schemes can be applied, in particular, to the stationary linear first
order equation in
${\Bbb R}^n$
. We present several
examples of schemes
belonging to this class and with fast convergence to the solution.
166 citations
••
01 Feb 2007
TL;DR: In this correspondence, adaptive critic approximate dynamic programming designs are derived to solve the discrete-time zero-sum game in which the state and action spaces are continuous, which results in a forward-in-time reinforcement learning algorithm that converges to the Nash equilibrium of the corresponding zero-Sum game.
Abstract: In this correspondence, adaptive critic approximate dynamic programming designs are derived to solve the discrete-time zero-sum game in which the state and action spaces are continuous. This results in a forward-in-time reinforcement learning algorithm that converges to the Nash equilibrium of the corresponding zero-sum game. The results in this correspondence can be thought of as a way to solve the Riccati equation of the well-known discrete-time Hinfin optimal control problem forward in time. Two schemes are presented, namely: 1) a heuristic dynamic programming and 2) a dual-heuristic dynamic programming, to solve for the value function and the costate of the game, respectively. An Hinfin autopilot design for an F-16 aircraft is presented to illustrate the results
162 citations
••
TL;DR: In this paper, the authors define a Stackelberg equilibrium with robust decision makers in which the leader and the follower have different worst-case models despite sharing a common approximating model.
162 citations
••
TL;DR: A verification theorem of variational inequality type is proved and is applied to solve explicitly some classes of optimal harvesting delay problems.
Abstract: We consider optimal harvesting of systems described by stochastic differential equations with delay. We focus on those situations where the value function of the harvesting problem depends on the initial path of the process in a simple way, namely through its value at 0 and through some weighted averages A verification theorem of variational inequality type is proved. This is applied to solve explicitly some classes of optimal harvesting delay problems
159 citations