scispace - formally typeset
Search or ask a question
Topic

Bellman equation

About: Bellman equation is a research topic. Over the lifetime, 5884 publications have been published within this topic receiving 135589 citations.


Papers
More filters
Journal ArticleDOI
TL;DR: A new method to approximate Markov perfect equilibrium in largescale Ericson and Pakes (1995)-style dynamic oligopoly models that are not amenable to exact solution due to the curse of dimensionality is introduced.
Abstract: In this article, we introduce a new method to approximate Markov perfect equilibrium in largescale Ericson and Pakes (1995)-style dynamic oligopoly models that are not amenable to exact solution due to the curse of dimensionality. The method is based on an algorithm that iterates an approximate best response operator using an approximate dynamic programming approach. The method, based on mathematical programming, approximates the value function with a linear combination of basis functions. We provide results that lend theoretical support to our approach. We introduce a rich yet tractable set of basis functions, and test our method on important classes of models. Our results suggest that the approach we propose significantly expands the set of dynamic oligopoly models that can be analyzed computationally.

52 citations

Posted Content
TL;DR: A new method is used to prove that the value functions are deterministic, satisfy the dynamic programming principle, and are viscosity solutions to the associated generalized Hamilton--Jacobi--Bellman (HJB) equations.
Abstract: In this paper we study stochastic optimal control problems of fully coupled forward-backward stochastic differential equations (FBSDEs). The recursive cost functionals are defined by controlled fully coupled FBSDEs. We study two cases of diffusion coefficients $\sigma$ of FSDEs. We use a new method to prove that the value functions are deterministic, satisfy the dynamic programming principle (DPP), and are viscosity solutions to the associated generalized Hamilton-Jacobi-Bellman (HJB) equations. The associated generalized HJB equations are related with algebraic equations when $\sigma$ depends on the second component of the solution $(Y, Z)$ of the BSDE and doesn't depend on the control. For this we adopt Peng's BSDE method, and so in particular, the notion of stochastic backward semigroup in [16]. We emphasize that the fact that $\sigma$ also depends on $Z$ makes the stochastic control much more complicate and has as consequence that the associated HJB equation is combined with an algebraic equation, which is inspired by Wu and Yu [19]. We use the continuation method combined with the fixed point theorem to prove that the algebraic equation has a unique solution, and moreover, we also give the representation for this solution. On the other hand, we prove some new basic estimates for fully coupled FBSDEs under the monotonic assumptions. In particular, we prove under the Lipschitz and linear growth conditions that fully coupled FBSDEs have a unique solution on the small time interval, if the Lipschitz constant of $\sigma$\ with respect to $z$ is sufficiently small. We also establish a generalized comparison theorem for such fully coupled FBSDEs.

52 citations

Journal ArticleDOI
TL;DR: In this article, the authors consider the case where the risk is a stock whose price process is a geometric Brownian motion and find a dynamic choice of the investment policy which minimizes the ruin probability of the company.
Abstract: We consider that the surplus of an insurance company follows a Cramer–Lundberg process. The management has the possibility of investing part of the surplus in a risky asset. We consider that the risky asset is a stock whose price process is a geometric Brownian motion. Our aim is to find a dynamic choice of the investment policy which minimizes the ruin probability of the company. We impose that the ratio between the amount invested in the risky asset and the surplus should be smaller than a given positive bound a . For instance the case a = 1 means that the management cannot borrow money to buy stocks. [Hipp, C., Plum, M., 2000. Optimal investment for insurers. Insurance: Mathematics and Economics 27, 215–228] and [Schmidli, H., 2002. On minimizing the ruin probability by investment and reinsurance. Ann. Appl. Probab. 12, 890–907] solved this problem without borrowing constraints. They found that the ratio between the amount invested in the risky asset and the surplus goes to infinity as the surplus approaches zero, so the optimal strategies of the constrained and unconstrained problems never coincide. We characterize the optimal value function as the classical solution of the associated Hamilton–Jacobi–Bellman equation. This equation is a second-order non-linear integro-differential equation. We obtain numerical solutions for some claim-size distributions and compare our results with those of the unconstrained case.

52 citations

Journal ArticleDOI
TL;DR: A version of the DICE-2007 model designed for uncertaintyanalysis is introduced, including a closed form continuous time approximation to the exogenous processes in DICE and a Bellman equation for DICE that disentangles risk attitude from the propensity to smooth consumption over time.
Abstract: We introduce a version of the DICE-2007 model designed for uncertainty analysis. DICE is a wide-spread deterministic integrated assessment model of climate change. Climate change, long-term economic development, and their interactions are highly uncertain. The quantitative analysis of optimal mitigation policy under uncertainty requires a recursive dynamic programming implementation of integrated assessment models. Such implementations are subject to the curse of dimensionality. Every increase in the dimension of the state space is paid for by a combination of (exponentially) increasing processor time, lower quality of the value or policy function approximations, and reductions of the uncertainty domain. The paper promotes a state reduced, recursive dynamic programming implementation of the DICE-2007 model. We achieve the reduction by simplifying the carbon cycle and the temperature delay equations. We compare our model's performance and that of the DICE model to the scientific AOGCM models emulated by MAGICC 6.0 and find that our simplified model performs equally well as the original DICE model. Our implementation solves the infinite planning horizon problem in an arbitrary time step. The paper is the first to carefully analyze the quality of the value function approximation using two different types of basis functions and systematically varying the dimension of the basis. We present the closed form, continuous time approximation to the exogenous (discretely and inductively defined) processes in DICE, and we present a numerically more efficient re-normalized Bellman equation that, in addition, can disentangle risk attitude from the propensity to smooth consumption over time.

52 citations

Journal ArticleDOI
TL;DR: In this article, a necessary condition in terms of lower directional Dini derivates of the value function is given, and a strengthened version of the necessary condition gives an optimal feedback control and a procedure for approximating optimal controls.
Abstract: Optimal control problems governed by ordinary differential equations with control constraints that are not necessarily compact are considered. Conditions imposed on the data and on the structure of the terminal sets imply that the minimum is attained and that the value function is locally Lipschitz. A necessary condition in terms of lower directional Dini derivates of the value function is given. The condition reduces to the Bellman–Hamilton–Jacobi (BHJ) condition at points of differentiability of the value, and for a subclass of the problems considered implies that the value is a viscosity solution of the BHJ equation. A strengthened version of the necessary condition gives an optimal feedback control and a procedure for approximating optimal controls.

52 citations


Network Information
Related Topics (5)
Optimal control
68K papers, 1.2M citations
87% related
Bounded function
77.2K papers, 1.3M citations
85% related
Markov chain
51.9K papers, 1.3M citations
85% related
Linear system
59.5K papers, 1.4M citations
84% related
Optimization problem
96.4K papers, 2.1M citations
83% related
Performance
Metrics
No. of papers in the topic in previous years
YearPapers
2023261
2022537
2021369
2020411
2019348
2018353