Showing papers on "Bellman equation published in 1984"
TL;DR: It is proved the rate of convergence of the approximate solutions to the value function of the original problem is of order 1 as the discretization step tends to zero, provided a semiconcavity assumption is satisfied.
Abstract: We consider an infinite horizon discounted optimal control problem and its time discretized approximation, and study the rate of convergence of the approximate solutions to the value function of the original problem. In particular we prove the rate is of order 1 as the discretization step tends to zero, provided a semiconcavity assumption is satisfied. We also characterize the limit of the optimal controls for the approximate problems within the framework of the theory of relaxed controls.
144 citations
TL;DR: In this article, the problem of determining optimal portfolio rules is considered, where prices are assumed to be stochastic processes of a fairly general nature, expressible as stochiastic integrals with respect to semimartingales.
111 citations
01 Jan 1984
TL;DR: In this paper, it was shown that the optimal value as a function of the parameters is directionally differentiable and the directional derivatives are expressed by a minimax formula which generalizes the one of Gol'shtein in convex programming.
Abstract: A parameterized nonlinear programming problem is considered in which the objective and constraint functions are twice continuously differentiable. Under the assumption that certain multiplier vectors appearing in generalized second-order necessary conditions for local optimality actually satisfy the weak sufficient condition for local optimality based on the augmented Lagrangian, it is shown that the optimal value in the problem, as a function of the parameters, is directionally differentiable. The directional derivatives are expressed by a minimax formula which generalizes the one of Gol’shtein in convex programming.
90 citations
TL;DR: A new equivalent formulation of Clarke's multiplier rule for nonsmooth optimization problems is given, which shows that the set of all multipliers satisfying necessary optimality conditions is the union of a finite number of closed convex cones.
Abstract: For several types of finite or infinite dimensional optimization problems the marginal function or optimal value function is characterized by different local approximations such as generalized gradients, generalized directional derivatives, directional Hadamard or Dini derivatives. We give estimates for these terms which are determined by multipliers satisfying necessary optimality conditions. When the functions which define the optimization problem are more than once continuously differentiable, then higher order necessary conditions are employed to obtain refined estimates for the marginal function. As a by-product we give a new equivalent formulation of Clarke's multiplier rule for nonsmooth optimization problems. This shows that the set of all multipliers satisfying these necessary conditions is the union of a finite number of closed convex cones.
55 citations
01 Jan 1984
TL;DR: In this paper, the relative importance of some assumptions in recent results concerning the continuity and directional differentiability of the optimal value function in a nonlinear programming problem is discussed, and examples and counterexamples are provided.
Abstract: This short paper illustrates by examples and counterexamples the relative importance of some assumptions in recent results concerning the continuity and the directional differentiability of the optimal value function in a nonlinear programming problem.
25 citations
TL;DR: In this paper, it is shown that decision and forecast horizons induce an alternate definition of optimality which is stronger than the concept of 'agreeable plan' proposed by Hammond.
19 citations
TL;DR: In this article, a non lineaire elliptique equation non-lineaire is proposed, which is a solution unique sous certain hypotheses, and a methode de programmation dynamique.
Abstract: L'etude de problemes de commande optimale stochastique de processus de diffusion controles a la fois par impulsions et de facon continue, et arretes a la frontiere d'une region bornee de R N , conduit par une methode de programmation dynamique, a une equation non lineaire elliptique. On montre que cette equation a une solution unique sous certaines hypotheses
16 citations
TL;DR: In this paper, the existence and uniqueness of the solution to the Bellman equation for ergodic control of one-dimensional diffusions is established under a "near-monotonicity" condition on the cost.
8 citations
TL;DR: A generalized concept summary of optimality is established, which unifies various related principles and may be viewed as a sort of complement and extension of Bellman's work.
Abstract: The optimization of a panweight network is studied. A generalized concept summary of optimality is established, which unifies various related principles. This may be viewed as a sort of complement and extension of Bellman's work. Some of its applications in dynamic programming are discussed. A sufficient condition and three necessary conditions are given under which Bellman's principle holds.
6 citations
TL;DR: In this article, the authors consider a linear system with additive noise in Hilbert space and minimize a convex functional associated with this process, and derive a necessary and sufficient condition for a control to be optimal by evaluating the subdifferential of the cost function.
Abstract: We consider a linear system with additive noise in Hilbert space and minimize a convex functional associated with this process. A necessary and sufficient condition for a control to be optimal is derived by evaluating the subdifferential of the cost function. Then the subdifferential of the value function is characterized. Finally using these results and a conditional value function, optimal controls are characterized as a feedback law in terms of the value function.
2 citations
DOI•
01 Feb 1984TL;DR: This work introduces a class of composite-source models as a means of representing consistently signals of this nature and demonstrates how the principle of optimality enables efficient algorithms for recursive maximum-likelihood estimation to be developed.
Abstract: Random processes of considerable importance in signal processing often exhibit short-term stationary statistical attributes although in the long term they appear to behave in a nonstationary manner. Image signals belong to this category. In this work we introduce a class of composite-source models as a means of representing consistently signals of this nature. A composite likelihood function is derived, the subsequent maximisation of which yields estimates of the parameters that are associated with the composite-source model. It is a fact, that maximisation of the composite likelihood function is almost intractable by analytical means. However, by introducing optimisation techniques based on dynamic programming, maximum-likelihood estimation of composite-source models is simplified drastically. A graph-theoretic approach is adopted to demonstrate how the principle of optimality enables efficient algorithms for recursive maximum-likelihood estimation to be developed. Algorithms applied for one-dimensional as well as two-dimensional signals are presented. In both cases it is shown that the estimation problem is equivalent to the problem of identifying the maximumlikelihood path which traverses a directed graph of specific structure. Finally, it is shown that composite source models so estimated can be used in image coding systems which require the least transmission rate for prespecified levels of average distortion of the transmitted image signals.
TL;DR: In this article, the authors extend a class of structural nonlinear one-predator one-prey models to multispecies aquatic ecosystem models and construct the relationship between aggregated (reduced order) models and the decomposed (non-interactive) models on the basis of a specific use of Bellman's principle of optimality.
TL;DR: Computational results confirm that, owing to the nature of the interaction between overlapping shift schedules, total enumeration was superior to dynamic programming and branch-and-bound integer programming for a bank-clerk scheduling problem.
Abstract: Dynamic programming computational efficiency rests upon the so-called principle of optimality, where it is possible to decompose combinatorial problems into individual sub-problems (stages). The sub-problems are solved independently and linked together through the use of the state variables which reflect optimal decisions for other (preceding) stages.
TL;DR: In this paper, the authors considered a control problem with one-dimensional Wiener plus independent Poisson noise and finite fuel, and sufficient conditions were given so that the Bellman equations have a modified bang-bang solution for the optimal control.
Abstract: This control problem has a one-dimensional Wiener plus independent Poisson noise with bounds on the control, as well as finite fuel (bounded variation of control). The object is to minimize the distance from the origin in accord with a specified symmetric function. Sufficient conditions are given so that the Bellman equations have a modified bang-bang solution for the optimal control. Fuel is used in accord with this control until either the time runs out, or the fuel is exhausted. The finite and infinite horizon cases are treated separately.
01 Dec 1984
TL;DR: In this paper, discrete Bellman equations and quasi-variational inequalities are written as fixed-point problems for an appropriate operator T. Under suitable assumptions, they show that either T or a power of T is a contraction.
Abstract: We write discrete Bellman equations and quasi-variational inequalities as fixed-point problems for an appropriate operator T. Under suitable assumptions, we show that either T or a power of T is a contraction.