Showing papers on "Dynamic programming published in 1975"
371 citations
TL;DR: The functional equation approach of dynamic programming is used to extend this model to the multiperiod case, and the structure of optimal ordering policies is analyzed.
Abstract: This paper deals with the problem of computing optimal ordering policies for a single product with a lifetime of exactly m periods. Costs are charged against ordering, holding, shortages, and out-dating. To take explicit account of the perishability, we substitute a cost to be incurred at the time of outdating. The functional equation approach of dynamic programming is used to extend this model to the multiperiod case, and the structure of optimal ordering policies is analyzed.
286 citations
TL;DR: In this article, a dynamic or multi-period location-allocation formulation is developed from the static problem of locating G facilities among M possible sites to serve N demand points, and a dynamic model provides a tool for analyzing tradeoffs among present values of static distribution costs in each period and costs of relocating facilities.
Abstract: A dynamic or multiperiod location-allocation formulation is developed from the static problem of locating G facilities among M possible sites to serve N demand points This dynamic model provides a tool for analyzing tradeoffs among present values of static distribution costs in each period and costs of relocating facilities The objective is to specify the plan for facility locations and relocations and for attendant allocations of demands which minimize these costs Two methods of solution are presented First, a mixed-integer programming approach is used to solve sample problems From computational results reported for structurally-similar problems, it seems that efficient general purpose codes for this method would be capable of solving problems with at least 5 periods, 5 potential sites, and 15 demand points The second method, dynamic programming, is capable of increasing the size of problems that are computationally feasible The dynamic programming approach is quite attractive when the relative va
205 citations
TL;DR: This short paper considers a discretization procedure often employed in practice and shows that the solution of the discretized algorithm converges to the Solution of the continuous algorithm, as theDiscretization grids become finer and finer.
Abstract: The computational solution of discrete-time stochastic optimal control problems by dynamic programming requires, in most cases, discretization of the state and control spaces whenever these spaces are infinite. In this short paper we consider a discretization procedure often employed in practice. Under certain compactness and Lipschitz continuity assumptions we show that the solution of the discretized algorithm converges to the solution of the continuous algorithm, as the discretization grids become finer and finer. Furthermore, any control law obtained from the discretized algorithm results in a value of the cost functional which converges to the optimal value of the problem.
191 citations
TL;DR: In this paper, a dynamic programming approach is presented to determine the allocation which minimizes the duration of the project (critical path) in order to solve the problem of time-cost tradeoff.
Abstract: To solve the problem of time-cost tradeoff in project management with available models, a choice must be made between heuristic approaches and algorithms based upon restrictive assumptions about the shape of the cost-time functions of the activities. The algorithm presented in this article involves a dynamic-programming approach to determine the allocation which minimizes the duration of the project (critical path). The main advantage of this model is its ability to determine the optimum allocation among activities with arbitrary cost-time functions. Also, computational shortcuts for functions with special properties can be used to increase the efficiency of the model. Objective functions of networks with special structures decompose into sequences of one-dimensional optimization problems. Although some complex networks have objective functions which cannot be fully decomposed, the dimensions of these functions are considerably less than the number of activities involved. If the activities have nonincreas...
163 citations
05 May 1975
TL;DR: A dynamic programming algorithm is presented which produces optimal code for any machine in the class; this algorithm runs in time which is linearly proportional to the number of vertices in an expression tree.
Abstract: We discuss the problem of generating code for a wide class of machines, restricting ourselves to the computation of expression trees. After defining a broad class of machines and discussing the properties of optimal programs on these machines, we derive a necessary and sufficient condition which can be used to prove the optimality of any code generation algorithm for expression trees on this class. We then present a dynamic programming algorithm which produces optimal code for any machine in the class; this algorithm runs in time which is linearly proportional to the number of vertices in an expression tree.
96 citations
TL;DR: A heuristic for the knapsack problem that recursively determines a solution by making a variable with smallest marginal unit cost as large as possible is analyzed.
Abstract: This paper analyzes a heuristic for the knapsack problem that recursively determines a solution by making a variable with smallest marginal unit cost as large as possible. Recursive necessary and sufficient conditions for the optimality of such “greedy” solutions and a “good” algorithm for verifying these conditions are given. Maximum absolute error for nonoptimal “greedy” solutions is also examined.
91 citations
TL;DR: This paper uses primal and dual bounds in a branch and bound strategy to develop a procedure for locating a small number of nearly optimal separation sequences; furthermore, the optimal sequence must be among those found.
70 citations
TL;DR: Conditions sufficient to prove that an optimal stationary strategy exists in a discounted stationary risk sensitive Markov decision process with constant risk aversion are identified and provided.
Abstract: Multi-stage decision processes are considered, in notation which is an outgrowth of that introduced by Denardo [Denardo, E. 1967. Contraction mappings in the theory underlying dynamic programming. SIAM Rev. 9 165–177.]. Certain Markov decision processes, stochastic games, and risk-sensitive Markov decision processes can be formulated in this notation. We identify conditions sufficient to prove that, in infinite horizon nonstationary processes, the optimal infinite horizon (present) value exists, is uniquely defined, is what is called “structured,” and can be found by solving Bellman's optimality equations: ϵ-optimal strategies exist: an optimal strategy can be found by applying Bellman's optimality criterion; and a specially identified kind of policy, called a “structured” policy is optimal in each stage. A link is thus drawn between (i) studies such as those of Blackwell [Blackwell, D. 1965. Discounted dynamic programming. Ann. Math. Stat. 36 226–235.] and Strauch [Strauch, R. 1966. Negative dynamic prog...
67 citations
TL;DR: A dynamic programming procedure is presented whereby an optimal solution to the problem may be obtained and a further approximate method based on the technique of successive approximations is shown to provide an alternative means for tackling large problems.
Abstract: The deterministic N-job two-machine flow-shop sequencing problem with finite intermediate storage is considered with the objective of minimizing the total processing time. A dynamic programming pro...
59 citations
TL;DR: The problems verifying the formulas include operations of single-and multiple-purpose reservoir networks and optimal design of storm sewer systems and aqueducts.
Abstract: The computer time required for a water resource optimization problem by dynamic programing (DP) or discrete differential dynamic programing (DDDP) may be considered as the sum of the compiling time for programing language translation, the initiating time for the program, and the execution time TE. The execution time is the dominant component of the total computer time and may be formulated as TE = TaMN Пi=1S Qi Пj=1D Pj, in which Ta is the time for one unit operation, M is the number of iterations involved in optimization, N is the number of stages, and Qi and Pj are the number of feasible values that state variable i (i = 1, 2, …, S) and decision variable j(j = 1,2,…, D), respectively, can take in each iteration or in the optimizational procedure. The value of Ta depends on nature of problem, type of computer, method of coding, kind of compiler, and other factors. The computer memory required for this optimization problem may be considered as the sum of the machine memory which is relatively constant, the code memory which increases slowly with the problem size, and the data memory which increases rapidly with the problem size. The data memory consists of the basic data memory, the performance data memory P = 2 Пi=1s Qi, and the optimal decision data memory T = ND Пi=1s Qi. The cost trade offs involved in replacing core memory by slow-speed memory (disks or tapes) are investigated. The computers used in the analysis include: IBM 360/50, IBM 360/75, IBM 360/91 and Burroughs B-6700. The problems verifying the formulas include operations of single-and multiple-purpose reservoir networks and optimal design of storm sewer systems and aqueducts.
01 Dec 1975
TL;DR: Incomplete dynamic programming (IDP) as mentioned in this paper is a method for improving the first cycle of the dynamic programming policy iteration approach for large-scale capacity expansion problems with many locations.
Abstract: Methods for planning capacity expansion typically have been restricted to problems with many locations in a static environment or a few locations in a dynamic environment. Two approaches are developed here for dynamic capacity planning problems with many locations. The first is an approximate approach based on an equivalent annual cost measure, and the second is a procedure for systematic improvement of the approximate solution. The method for improvement is called “incomplete dynamic programming” since it consists of an approximation to the first cycle of the dynamic programming policy iteration approach. Computational results are reported for tests of the methods against dynamic programming solutions for small problems. Applications are made to two versions of a large-scale problem of planning capacity expansion for India's nitrogenous fertilizer industry, and results are compared with those for other approaches.
TL;DR: In this paper, a global optimal solution of a number of different problems in respect to stratification and grouping of random variables or their values result in optimization problems of the same structure.
Abstract: In statistics and their fields of application a number of different problems in respect to stratification and grouping of random variables or their values result in optimization problems of the same structure. By a suitable transformation a global optimal solution of these problems can be determined by dynamic programming. The results are illustrated for discrete and continuous random variables by numerical results.
TL;DR: A two-dimensional dynamic programming problem is posed and by relaxing some of the restraints on the problem it is reduced to a standardynamic programming problem.
Abstract: A two-dimensional dynamic programming problem is posed. By relaxing some of the restraints on the problem it is reduced to a standard dynamic programming problem. Results are quoted from a particular case study.
TL;DR: In this article, quantitative guidelines which discriminate between dynamic programing models and policy iteration models are developed, and a judgment that stochastic dynamic programs should be the preferred algorithm is reached.
Abstract: Linear programing formulations, dynamic programing formulations, and policy iteration models have all been used to solve for optimal operating rules for a single stochastic reservoir. In this paper, quantitative guidelines which discriminate between dynamic programing models and policy iteration models are developed. A judgment that stochastic dynamic programing models should be the preferred algorithm is reached.
TL;DR: The purpose is to illustrate that one should be invoking the optimality equations and/or the Optimality criterion, rather than the Principle of Optimality, in analyzing dynamic models.
Abstract: The Principle of Optimality is examined informally in the context of discounted Markov decision processes. Our purpose is to illustrate that one should be invoking the optimality equations and/or the optimality criterion, rather than the Principle of Optimality in analyzing dynamic models. A counterexample to one interpretation of the Principle is given. It involves a foolish action at the second stage from a state that can be reached, but with probability zero. Redefining optimality as in Hinderer [Hinderer, K. 1970. Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter. Springer-Verlag, New York.], restores the Principle, at the cost of a weaker notion of optimality.
TL;DR: It is shown that the optimal betting strategy for a continuous model of backgammon is to double when you have an 80 percent chance of winning.
Abstract: This paper shows that the optimal betting strategy for a continuous model of backgammon is to double when you have an 80% chance of winning. We discuss the differences with the published literature on the real game and the problem of infinite expectations. The optimal strategy for a simulation of the endgame is computed by dynamic programming.
01 Jan 1975
TL;DR: Under a condition of Liapunov function type the Laurent expansion of the total discounted expected return for the various policies is derived and the equivalence of the sensitive optimality criteria as introduced by Veinott is shown.
Abstract: Discrete time Markov decision processes with a countable state space are investigated. Under a condition of Liapunov function type the Laurent expansion of the total discounted expected return for the various policies is derived. Moreover, the equivalence of the sensitive optimality criteria as introduced by Veinott is shown.
TL;DR: The existence of optimal plans is established using results of non-stationary dynamic programming and general criteria of optimality are derived.
TL;DR: An algorithm for transposing large rectangular matrices using optimal partitioning and powers-of-2 partitioning is presented and it is found in the cases tested that a speed gain of up to 5.55 is obtained by optimizing the partition sizes.
Abstract: The purpose of this paper is to present an algorithm for transposing large rectangular matrices. This is basically a generalization of Eklundh's algorithm. Eklundh's method is designed to handle arrays in place of using powers-of-2 approach wherein partitions of the matrix have dimensions which are powers of 2. The algorithm presented here does not have this restriction. The choice of partitions is posed as an optimal control problem amenable to solution using Bellman's principle of optimality. The optimal partitioning of the matrix can be determined depending on the computer configuration (i.e., core size, whether moving-head or fixed-head disk drives are used, number of words per track, etc.) and the matrix dimensions. Simple modifications are noted for transposing complex/double precision and packed integer matrices. Experimental results are presented comparing the times needed for transposition using optimal partitioning and powers-of-2 partitioning. It is found in the cases tested that a speed gain of up to 5.55 is obtained by optimizing the partition sizes.
TL;DR: An exact method for solving all-integer non-linear programming problems with a separable nondecreasing objective function is presented in this article, which is used to efficiently search candidate hypersurfaces for the optimal feasible integer solution.
Abstract: An exact method for solving all-integer non-linear programming problems with a separable non-decreasing objective function is presented. Dynamic programming methodology is used to efficiently search candidate hypersurfaces for the optimal feasible integer solution. An efficient computational and storage scheme exists and initial calculations give very promising results.
TL;DR: In this paper, a facility stage discrete differential dynamic programming approach is presented for determining the optimal operating policy for a multiple purpose multiple reservoir system, where stages in the dynamic programming formulation are defined to represent reservoirs and state variables are represented to represent the amounts of release from reservoirs.
Abstract: A facility stage discrete differential dynamic programming approach is presented for determining the optimal operating policy for a multiple purpose multiple reservoir system. The stages in the dynamic programming formulation are defined to represent reservoirs and state variables are defined to represent the amounts of release from reservoirs. Discrete differential dynamic programming, which is an iterative technique, is used to solve the dynamic programming problem to determine the optimal operating policy. The optimal operating policy is the policy which provides the maximum returns for the system. The approach is applied to an example problem which has been solved by gradient projection and conjugate gradient techniques.
TL;DR: A rule of thumb avoiding a computer program and an example are given, which will be used to maximize the dynamic range and the noise distance of the filters.
Abstract: RC-active filters are usually realized by cascading 2nd order stages. The degrees-of-freedom in this procedure are: the pole-zero assignments to form the second order transfer functions, the gain-factors for the stages and the sequence in the cascade. These freedoms will simultaneously be used to maximize the dynamic range and the noise distance of the filters. The exact solution to the problem will be obtained by dynamic programming. A rule of thumb avoiding a computer program and an example are given.
01 Jan 1975
Book•
01 Jan 1975
TL;DR: This chapter discusses Dynamic Programming under Risk, which combines Risk and Multistage Optimization with Quadratic Programming, and the Recursion Procedure in Dynamic Programming.
Abstract: I. Linear and Nonlinear Programming.- II. Elements of the Mathematical Theory of Nonlinear Programming.- A. Constrained Optimization.- B. Kuhn-Tucker Optimization.- III. Linearization of Nonlinear Programming Problems.- A. Linear Approximations and Linear Programming.- B. Partitioning of Variables.- C. Separable Programming.- IV. Quadratic Programming.- A. Linear and Quadratic Programming.- B. The Kuhn-Tucker Conditions.- C. Combinatorial Solution.- D. Wolfe's Method.- E. The Simplex Method for Quadratic Programming.- F. Beale's Method.- G. Computer Solution.- H. Some Industrial Applications.- V. Dynamic Programming and Multistage Optimization.- VI. Applications of Dynamic Programming.- A. The Shortest Path through a Network.- B. Production Planning.- C. Inventory Problems.- D. Investment Planning.- E. Allocation of Salesmen.- F. Cargo Loading and the Knapsack Problem.- VII. Several Decision and State Variables.- VIII. Infinite-Stage Problems.- IX. Dynamic Programming under Risk.- A. Risk and Multistage Optimization.- B. Dynamic Programming and Markov Processes.- X. Appendix: The Recursion Procedure in Dynamic Programming.- A. Stage Optimization.- B. Backward Recursion for Given Initial State.- C. Forward Recursion for Given Initial State.- D. Forward Recursion for Given Final State.- E. Backward Recursion for Given Final State.- F. Given Initial and Final States.- Answers to Exercises.- References.
TL;DR: In this paper, the optimal design of elastic trusses from a dynamic programming point of view is discussed, and the functional equation approach is shown to furnish a direct solution to the problem of determining a design among all possible ones satisfying certain volume and displacement constraints, for which the maximum stress is a minimum.
Abstract: The optimal design of elastic trusses is discussed from a dynamic programming point of view. Emphasis is placed on minimum volume design of statically determinate trusses with displacement and stress constraints in the discrete case, i.e., when the cross-sectional areas of the bars are available from a discrete set of values. This, a design constraint usually very difficult to handle with standard nonlinear programming algorithms, is naturally incorporated in the present formulation. In addition, the functional equation approach is shown to furnish a direct solution to the problem of determining a design, among all possible ones satisfying certain volume and displacement constraints, for which the maximum stress is a minimum. A successive approximation approach is briefly indicated as an extension of the method to solve statically indeterminate trusses. Finally, several numerical examples are presented and the main features of the methods are briefly exposed.
Proceedings Article•
03 Sep 1975
TL;DR: A general heuristic search algorithm with estimate is given, which is a nontrivial extension of algorithm A, which can be simplified until the classical version, with additive cost functions, is reached.
Abstract: In this paper we approach, using artificial intelligence methods, the problem of finding a minimal-cost path in a functionally weighted graph, i.e a graph with monotone cost functions associated with the arcs This problem is important since solving any system of functional equations in a general dynamic programming formulation can be shown equivalent to it. A general heuristic search algorithm with estimate is given, which is a nontrivial extension of algorithm A. by Hart, Nilsson and Raphael. Putting some constraints on cost functions and on the estimate, this algorithm can be simplified until the classical version, with additive cost functions, is reached.
01 Jan 1975
TL;DR: In this paper, the results of Chow, Moriguti, Robbins and Samuels (CMRS) for the secretary problem have been extended to the case with interview costs. And the solution to the problem with zero interview cost is obtained.
Abstract: : The results of Chow, Moriguti, Robbins and Samuels (CMRS) (1964) for the secretary problem have been extended to the case with interview costs. Certain tables for chosen cost functions and certain n(n denotes the total number of candidates) have been provided illustrating the optimal rule and the optimal expected pay-off. A couple of approximations which involve solving differential equations are also given. Next is considered the secretary problem when one can recall the immediately preceding candidate. The optimal rule is derived and is ullustrated for certain cost functions and a few chosen values of n. The approximating differential equations are obtained. The solution to the problem with zero interview cost is obtained.