Bellman equation

About: Bellman equation is a research topic. Over the lifetime, 5884 publications have been published within this topic receiving 135589 citations.

A 4-stated DICE: quantitatively addressing uncertainty effects in climate change

Abstract: We introduce a version of the DICE-2007 model designed for uncertainty analysis. DICE is a wide-spread deterministic integrated assessment model of climate change. Climate change, long-term economic development, and their interactions are highly uncertain. The quantitative analysis of optimal mitigation policy under uncertainty requires a recursive dynamic programming implementation of integrated assessment models. Such implementations are subject to the curse of dimensionality. Every increase in the dimension of the state space is paid for by a combination of (exponentially) increasing processor time, lower quality of the value or policy function approximations, and reductions of the uncertainty domain. The paper promotes a state-reduced, recursive dynamic programming implementation of the DICE-2007 model. We achieve the reduction by simplifying the carbon cycle and the temperature delay equations. We compare our model’s performance and that of the DICE model to the scientific AOGCM models emulated by MAGICC 6.0 and find that our simplified model performs equally well as the original DICE model. Our implementation solves the infinite planning horizon problem in an arbitrary time step. The paper is the first to carefully analyze the quality of the value function approximation using two different types of basis functions and systematically varying the dimension of the basis. We present the closed form, continuous time approximation to the exogenous (discretely and inductively defined) processes in DICE, and we present a numerically more efficient re-normalized Bellman equation that, in addition, can disentangle risk attitude from the propensity to smooth consumption over time.

e-optimality for bicriteria programs and its application to minimum cost flows

Abstract: A subsetS⊂X of feasible solutions of a multicriteria optimization problem is called e-optimal w.r.t. a vector-valued functionf:X→Y $$ \subseteq $$ ℝ K if for allx∈X there is a solutionz x∈S so thatf k(z x)≤(1+e)f k (x) for allk=1,...,K. For a given accuracy e>0, a pseudopolynomial approximation algorithm for bicriteria linear programming using the lower and upper approximation of the optimal value function is given. Numerical results for the bicriteria minimum cost flow problem on NETGEN-generated examples are presented.

On Optimal Harvesting Problems in Random Environments

Abstract: This paper investigates the optimal harvesting strategy for a single species living in random environments whose growth is given by a regime-switching diffusion. Harvesting acts as a (stochastic) control on the size of the population. The objective is to find a harvesting strategy which maximizes the expected total discounted income from harvesting {\em up to the time of extinction} of the species; the income rate is allowed to be state- and environment-dependent. This is a singular stochastic control problem with both the extinction time and the optimal harvesting policy depending on the initial condition. One aspect of receiving payments up to the random time of extinction is that small changes in the initial population size may significantly alter the extinction time when using the same harvesting policy. Consequently, one no longer obtains continuity of the value function using standard arguments for either regular or singular control problems having a fixed time horizon. This paper introduces a new sufficient condition under which the continuity of the value function for the regime-switching model is established. Further, it is shown that the value function is a viscosity solution of a coupled system of quasi-variational inequalities. The paper also establishes a verification theorem and, based on this theorem, an $\varepsilon$-optimal harvesting strategy is constructed under certain conditions on the model. Two examples are analyzed in detail.

Curse of dimensionality reduction in max-plus based approximation methods: Theoretical estimates and improved pruning algorithms

Abstract: Max-plus based methods have been recently developed to approximate the value function of possibly high dimensional optimal control problems. A critical step of these methods consists in approximating a function by a supremum of a small number of functions (max-plus “basis functions”) taken from a prescribed dictionary. We study several variants of this approximation problem, which we show to be continuous versions of the facility location and k-center combinatorial optimization problems, in which the connection costs arise from a Bregman distance. We give theoretical error estimates, quantifying the number of basis functions needed to reach a prescribed accuracy. We derive from our approach a refinement of the curse of dimensionality free method introduced previously by McEneaney, with a higher accuracy for a comparable computational cost.

Finite Horizon Optimal Investment and Consumption with Transaction Costs

Abstract: This paper concerns continuous-time optimal investment and consumption decision of a CRRA investor who faces proportional transaction costs and finite time horizon. In the no consumption case, it has been studied by Liu and Loewenstein (2002) and Dai and Yi (2006). Mathematically, it is a singular stochastic control problem whose value function satisfies a parabolic variational inequality with gradient constraints. The problem gives rise to two free boundaries which stand for the optimal buying and selling strategies, respectively. We present an analytical approach to analyze the behaviors of free boundaries. The regularity of the value function is studied as well. Our approach is essentially based on the connection between singular control and optimal stopping, which is first revealed to the present problem.