scispace - formally typeset
Search or ask a question
Topic

Bellman equation

About: Bellman equation is a research topic. Over the lifetime, 5884 publications have been published within this topic receiving 135589 citations.


Papers
More filters
Journal ArticleDOI
TL;DR: Monotonicity results for optimal policies of various queueing and resource sharing models are studied by concentrating on the events and the form of the value function instead of on thevalue function itself.
Abstract: In this paper we study monotonicity results for optimal policies of various queueing and resource sharing models. The standard approach is to propagate, for each specific model, certain properties of the dynamic programming value function. We propose a unified treatment of these models by concentrating on the events and the form of the value function instead of on the value function itself. This is illustrated with the systematic treatment of one and two-dimensional models.

169 citations

Journal ArticleDOI
TL;DR: In this article, a general method for constructing high-order approximation schemes for Hamilton-Jacobi-Bellman equations is given, based on a discrete version of the Dynamic Programming Principle.
Abstract: A general method for constructing high-order approximation schemes for Hamilton-Jacobi-Bellman equations is given. The method is based on a discrete version of the Dynamic Programming Principle. We prove a general convergence result for this class of approximation schemes also obtaining, under more restrictive assumptions, an estimate in $L^\infty$ of the order of convergence and of the local truncation error. The schemes can be applied, in particular, to the stationary linear first order equation in ${\Bbb R}^n$ . We present several examples of schemes belonging to this class and with fast convergence to the solution.

166 citations

Journal ArticleDOI
01 Feb 2007
TL;DR: In this correspondence, adaptive critic approximate dynamic programming designs are derived to solve the discrete-time zero-sum game in which the state and action spaces are continuous, which results in a forward-in-time reinforcement learning algorithm that converges to the Nash equilibrium of the corresponding zero-Sum game.
Abstract: In this correspondence, adaptive critic approximate dynamic programming designs are derived to solve the discrete-time zero-sum game in which the state and action spaces are continuous. This results in a forward-in-time reinforcement learning algorithm that converges to the Nash equilibrium of the corresponding zero-sum game. The results in this correspondence can be thought of as a way to solve the Riccati equation of the well-known discrete-time Hinfin optimal control problem forward in time. Two schemes are presented, namely: 1) a heuristic dynamic programming and 2) a dual-heuristic dynamic programming, to solve for the value function and the costate of the game, respectively. An Hinfin autopilot design for an F-16 aircraft is presented to illustrate the results

162 citations

Journal ArticleDOI
TL;DR: In this paper, the authors define a Stackelberg equilibrium with robust decision makers in which the leader and the follower have different worst-case models despite sharing a common approximating model.

162 citations

Journal ArticleDOI
TL;DR: A verification theorem of variational inequality type is proved and is applied to solve explicitly some classes of optimal harvesting delay problems.
Abstract: We consider optimal harvesting of systems described by stochastic differential equations with delay. We focus on those situations where the value function of the harvesting problem depends on the initial path of the process in a simple way, namely through its value at 0 and through some weighted averages A verification theorem of variational inequality type is proved. This is applied to solve explicitly some classes of optimal harvesting delay problems

159 citations


Network Information
Related Topics (5)
Optimal control
68K papers, 1.2M citations
87% related
Bounded function
77.2K papers, 1.3M citations
85% related
Markov chain
51.9K papers, 1.3M citations
85% related
Linear system
59.5K papers, 1.4M citations
84% related
Optimization problem
96.4K papers, 2.1M citations
83% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
2023261
2022537
2021369
2020411
2019348
2018353