Bellman equation

About: Bellman equation is a research topic. Over the lifetime, 5884 publications have been published within this topic receiving 135589 citations.

Necessary Optimality Conditions for Multiobjective Bilevel Programs

Abstract: The multiobjective bilevel program is a sequence of two optimization problems, with the upper-level problem being multiobjective and the constraint region of the upper level problem being determined implicitly by the solution set to the lower-level problem. In the case where the Karush-Kuhn-Tucker (KKT) condition is necessary and sufficient for global optimality of all lower-level problems near the optimal solution, we present various optimality conditions by replacing the lower-level problem with its KKT conditions. For the general multiobjective bilevel problem, we derive necessary optimality conditions by considering a combined problem, with both the value function and the KKT condition of the lower-level problem involved in the constraints. Most results of this paper are new, even for the case of a single-objective bilevel program, the case of a mathematical program with complementarity constraints, and the case of a multiobjective optimization problem.

Optimistic value model of uncertain optimal control

Abstract: Optimal control is an important field of study both in theory and in applications. Based on uncertainty theory, an expected value model of uncertain optimal control problem was studied by Zhu. In this paper, an optimistic value model for uncertain optimal control problem is investigated. Applying Bellman's principle of optimality, the principle of optimality for the model is presented. And then the equation of optimality is obtained for the optimistic value model of uncertain optimal control. Finally, a portfolio selection problem is solved by this equation of optimality.

Viscosity Solutions of Fully Nonlinear Equations

Abstract: : The eight publications produced by the project established a number of basic results in the theory of viscosity solutions of fully nonlinear differential equations of first and second order in finite and infinite dimensions. These equations arise in the dynamic programming theory of control and differential games (the finite dimensional theory for ode and the infinite dimensional theory for pde dynamics). Being fully nonlinear, the equations do not typically admit regular or classical solutions, and the appropriate notion is that of viscosity solutions. Two major advances in the first order infinite dimensional case consisted of determining the precise notion appropriate to a class of infinite dimensional problems with unbounded terms arising from the pde dynamics, and the examination of a limit case in which the value function is not a solution, but the maximal subsolution. Significant contributions to the second order theory include a new exposition of the finite dimensional theory based on results from previous funding, an infinite dimensional generalization of the foundational result used in this exposition, and the extension of the theory to second order equations in infinite dimensions with unbounded first order terms.

Optimal impulse control for a multidimensional cash management system with generalized cost functions

Abstract: We consider the optimal control of a multidimensional cash management system where the cash balances fluctuate as a homogeneous diffusion process in R n . We formulate the model as an impulse control problem on an unbounded domain with unbounded cost functions. Under general assumptions we characterize the value function as a weak solution of a quasi-variational inequality in a weighted Sobolev space and we show the existence of an optimal policy. Moreover we prove the local uniform convergence of a finite element scheme to compute numerically the value function and the optimal cost. We compute the solution of the model in two-dimensions with linear and distance cost functions, showing what are the shapes of the optimal policies in these two simple cases. Finally our third numerical experiment computes the solution in the realistic case of the cash concentration of two bank accounts made by a centralized treasury.

Approximating the optimal groundwater pumping policy in a multiaquifer stochastic conjunctive use setting

Abstract: This paper presents two methods for approximating the optimal groundwater pumping policy for several interrelated aquifers in a stochastic setting that also involves conjunctive use of surface water. The first method employs a policy iteration dynamic programming (DP) algorithm where the value function is estimated by Monte Carlo simulation combined with curve-fitting techniques. The second method uses a Taylor series approximation to the functional equation of DP which reduces the problem, for a given observed state, to solving a system of equations equal in number to the aquifers. The methods are compared using a four-state variable, stochastic dynamic programming model of Madera County, California. The two methods yield nearly identical estimates of the optimal pumping policy, as well as the steady state pumping depth, suggesting that either method can be used in similar applications.