Bellman equation

About: Bellman equation is a research topic. Over the lifetime, 5884 publications have been published within this topic receiving 135589 citations.

Viscosity solutions to parabolic master equations and McKean–Vlasov SDEs with closed-loop controls

Abstract: The master equation is a type of PDE whose state variable involves the distribution of certain underlying state process. It is a powerful tool for studying the limit behavior of large interacting systems, including mean field games and systemic risk. It also appears naturally in stochastic control problems with partial information and in time inconsistent problems. In this paper we propose a novel notion of viscosity solution for parabolic master equations, arising mainly from control problems, and establish its wellposedness. Our main innovation is to restrict the involved measures to a certain set of semimartingale measures which satisfy the desired compactness. As an important example, we study the HJB master equation associated with the control problems for McKean–Vlasov SDEs. Due to practical considerations, we consider closed-loop controls. It turns out that the regularity of the value function becomes much more involved in this framework than the counterpart in the standard control problems. Finally, we build the whole theory in the path dependent setting, which is often seen in applications. The main result in this part is an extension of Dupire’s (2009) functional Ito formula. This Ito formula requires a special structure of the derivatives with respect to the measures, which was originally due to Lions in the state dependent case. We provided an elementary proof for this well known result in the short note (2017), and the same arguments work in the path dependent setting here.

Some weak self-adjoint Hamilton–Jacobi–Bellman equations arising in financial mathematics

Abstract: In Gandarias (2011) [12] one of the present authors has introduced the concept of weak self-adjoint equations. This definition generalizes the concept of self-adjoint and quasi self-adjoint equations that were introduced by Ibragimov (2006) [11] . In this paper we find a class of weak self-adjoint Hamilton–Jacobi–Bellman equations which are neither self-adjoint nor quasi self-adjoint. By using a general theorem on conservation laws proved in Ibragimov (2007) [9] and the new concept of weak self-adjointness (Gandarias, 2011) [12] we find conservation laws for some of these partial differential equations.

Existence of solutions for the nonlinear partial differential equation arising in the optimal investment problem

Abstract: We are concerned with the solvablity of certain nonlinear partial differential equation (PDE), which is derived from the optimal investment problem under the random risk process The equation describes the evolution of the Arrow-Pratt coefficient of absolute risk aversion with respect to the optimal value function Employing the fixed point approach combined with the convergence argument we show the existence of solutions

Solving Stochastic Games

Abstract: Solving multi-agent reinforcement learning problems has proven difficult because of the lack of tractable algorithms. We provide the first approximation algorithm which solves stochastic games with cheap-talk to within ∊ absolute error of the optimal game-theoretic solution, in time polynomial in 1/∊. Our algorithm extends Murray's and Gordon's (2007) modified Bellman equation which determines the set of all possible achievable utilities; this provides us a truly general framework for multi-agent learning. Further, we empirically validate our algorithm and find the computational cost to be orders of magnitude less than what the theory predicts.

Branch-and-Sandwich: a deterministic global optimization algorithm for optimistic bilevel programming problems. Part II: Convergence analysis and numerical results

Abstract: In the first part of this work, we presented a global optimization algorithm, Branch-and-Sandwich, for optimistic bilevel programming problems that satisfy a regularity condition in the inner problem (Kleniati and Adjiman in J Glob Optim, 2014). The proposed approach can be interpreted as the exploration of two solution spaces (corresponding to the inner and the outer problems) using a single branch-and-bound tree, where two pairs of lower and upper bounds are computed: one for the outer optimal objective value and the other for the inner value function. In the present paper, the theoretical properties of the proposed algorithm are investigated and finite $$\varepsilon $$ � -convergence to a global solution of the bilevel problem is proved. Thirty-four problems from the literature are tackled successfully.