Showing papers on "Dynamic programming published in 1970"

Surrogate mathematical programming

Abstract: This paper presents an approach, similar to penalty functions, for solving arbitrary mathematical programs. The surrogate mathematical program is a lesser constrained problem that, in some cases, may be solved with dynamic programming. The paper deals with the theoretical development of this surrogate approach.

A Multistage Solution of the Template-Layout Problem

Abstract: The template-layout problem is to determine how to cut irregular-shaped two-dimensional pieces out of given stock sheets in an optimum manner without making an exhaustive search of all possible arrangements of the pieces. An algorithm is described for solving template-layout problems with a digital computer. The method of solution requires that the irregular shapes be enclosed, singly or in combination, in minimum area rectangles called modules. Individual modules will contain from one to perhaps eight optimally fitted irregular pieces. The modules are then packed into the given stock sheet(s) so as to optimize a specified objective function. The packing is carried out with a dynamic programming algorithm, which converts the multivariable problem into a multistage one. Successive iterations of the algorithm are used to determine whether higher order modules (containing more irregular-shaped pieces) improve the solution. A detailed description of the algorithm is given. An illustrative example is included and its computer solution is described. The paper concludes with an extension of the algorithm to an improved version which can be expected to yield solutions more closely approaching the true optimum.

Techniques for Optimum Spares Allocation: A Tutorial Review

Abstract: In many modern complex systems the problem of achieving high reliability leads to the use of interchangeable modular components accompanied by a stock of spare parts. This paper examines, compares, and assesses several of the techniques presented in the literature for allocating the numbers of spares of each part type to be stocked in order to maximize the system reliability subject to constraints on resources (i.e., weight, volume, cost, etc.). The problem of optimum spares allocation is complicated since resources are consumed in a discrete fashion and the expression for the system reliability is a nonlinear transcendental function. The classical dynamic programming algorithm produces all optimal spares allocations; however, the solution can become computationally intractable even with the aid of a modern high-speed digital computer. In the case of multiple constraints the time problem is vastly exacerbated. In such a case one must turn to a procedure that yields a near-optimal solution in a reasonable amount of computer time. Two approximate methods discussed in this paper are the incremental reliability per pound algorithm and the Lagrange multiplier algorithm. These algorithms are readily adaptable to handle multiple constraints. Computer programs are developed for each of the three optimization algorithms and are utilized to obtain the spares allocation for a few systems. The optimization theory presented is directly applicable to series or parallel systems. A concluding example illustrates how this can be extended to certain series-parallel systems.

Markov Models for Flow Regulation

Abstract: The problem of defining alternative operating policies for a single multipurpose reservoir was examined through the use of several pairs of discrete stochastic linear- and dynamic-programming models. The net flows into the reservoir were assumed to be serially correlated, their probabilistic sequence defined by first-order Markov chains. Each linear programming model was shown to correspond to a dynamic programming model. The solutions and computational efficiencies of each of the models were compared using a simplified numerical example based on an actual reservoir operating problem. Although the policies obtained from each pair of corresponding models were identical, the time required to solve the dynamic programming models was less than that required for the linear programming models.

A discrete-time differential dynamic programming algorithm with application to optimal orbit transfer

Abstract: : Recently, the notion of Differential Dynamic Programming has been used to obtain new second-order algorithms for solving non-linear optimal control problems. (Unlike conventional Dynamic Programming, the Principle of Optimality is applied in the neighborhood of a nominal, non-optimal, trajectory.) A novel feature of these algorithms is that they permit strong variations in the system trajectory. In this paper, Differential Dynamic Programming is used to develop a second-order algorithm for solving discrete-time dynamic optimization problems with terminal constraints. This algorithm also utilizes strong variations and, as a result, has certain advantages over existing discrete-time methods. A non-linear computed example is presented, and comparisons are made with the results of other researchers who have solved this problem. The experience gained during the computation has suggested some extensions to an earlier, previously published Differential Dynamic Programming algorithm for continuous time problems. These extensions, and their implications are discussed. (Author)

a Comparison of Some Dynamic, Linear and Policy Iteration Methods for Reservoir Operation

Abstract: Within the past few years, a number of papers have been published in which stochastic mathematical programming models, incorporating first order Markov chains, have been used to derive alternative sequential operating policies for a multiple purpose reservoir. This paper attempts to review and compare three such mathematical modeling and solution techniques, namely dynamic programming, policy iteration, and linear programming. It is assumed that the flows into the reservoir are serially correlated stochastic quantities. The design parameters are assumed fixed, i.e., the reservoir capacity and the storage and release targets, if any, are predetermined. The models are discrete since the continuous variables of time, volume, and flow are approximated by discrete units. The problem is to derive an optimal operating policy. Such a policy defines the reservoir release as a function of the current storage volume and inflow. The form of the solution and some of the advantages, limitations and computational efficiencies of each of the models and their algorithms are compared using a simplified numerical example.

A two‐stage optimization model for design of a multipurpose reservoir

Abstract: A mathematical model that can be useful in determining the optimum design and operation of a single multipurpose reservoir is presented. The development purposes include firm water supply, firm on-peak and dump hydropower production, flood control, and low flow augmentation. The model is based upon a two-stage optimization technique. It takes into account the fact that economic returns from a project are a function of both design and operational rules of the project. A dynamic programing algorithm which has a physical recursive equation computes the optimum operation policy of a feasible design. An iterative uniform grid sampling algorithm is then used to compare designs for which optimum operations are already determined and to select the best design. The model is applied to a hypothetical water resources project for illustrative purposes. The limitations of this mathematical model and some future research that may eliminate these limitations are discussed.

Addendum to Stankard and Gupta's Note on Lot Size Scheduling

Abstract: This paper presents a grouping procedure which yields improved solutions to Bomberger's Lot Size Scheduling Problem [1] The group definition is a generalization of the group definition proposed by Stankard and Gupta [3] Moreover, the generalization leads to a pseudo dynamic programming procedure which is readily programmable Computational experience has shown the procedure to be extremely efficient

System Optimization of Waste Treatment Plant Process Design

Abstract: An optimization model which integrates dynamic programming techniques into existing process design principles is presented. Process design of waste treatment facilities should consider all unit processes at the same time for overall system optimization. Designing and optimizing single processes individually has been the usual approach for many years. This method may yield the best design for a certain unit process but not necessarily the least-cost design in a multistage system. The optimization procedure for a serial, multistage system with two point boundary values is utilized. Following the mathematical development of the optimization model, an illustrative example is presented to demonstrate its application. For secondary treatment processes, two alternative subprocesses are considered simultaneously. A unique decision variable and typical state variables are used for each process. Representative unit process cost functions are also presented in the application.

Optimization of Series-Parallel-Series Networks

Abstract: This paper introduces a generalization of the frequently discussed problem of finding the optimum redundancy that maximizes the reliability of a network of components. Past work has restricted consideration to arrangements of redundant components called series-parallel networks. This paper allows a much broader class of arrangements called series-parallel-series networks. It is important to consider such arrangements for realistic situations in which components have more than one failure mode, or the combination of parallel paths introduces a failure probability. A dynamic programming algorithm is used to solve the more general problem for the case in which there are no constraints on the optimum solution. The algorithm is extended to handle multiple constraints using dominance and a variety of elimination methods to reduce the storage required in a computer implementation of the algorithm. Problems with as many as 15 serial components and three constraints have been solved with reasonable digital compute...

Jointly Optimal Deterministic Inventory and Replacement Policies

Abstract: A deterministic, continuous time, nonstationary inventory and replacement system model is formulated to find the number of orders, the order quantities, the times at which orders should be placed, and the replacement intervals which minimize the cost of operating, maintaining, and replacing a piece of equipment over a finite time horizon. Conditions are given for when the optimal policy is to buy equipments only when the last inventoried item is ready for replacement. A dynamic programming algorithm is presented for solving this problem. The structure of optimal policies is examined for the stationary case. Finally a computational procedure is given for solving the stationary, infinite horizon case.

Necessary and Sufficient Conditions for Dynamic Programming of Combinatorial Type

Abstract: A general formulation of discrete deterministic dynamic programming is given. This definition is obtained formally by derivation of a simplified algorithm from a general algorithm, and gives simultaneously the class of concurrent problems.

Optimal Reservoir Releases for Water Quality Control

Abstract: A general mathematical model for determining optimal flow release sequences for water quality control from multiple reservoir systems is presented. The model defines an optimal release sequence which provides either the “best quality of water” or the “least cost of reservoir storage” for a given stream flow at a particular point in the system. Using the converging-branch multistage system of dynamic programming, the flow release problem is decomposed into a sequence of subproblems or stages. At each stage or decision point, which represents a confluence of two regulated streams, optimal release sequences are obtained by an efficient enumeration process. The flow release model incorporates formulations for BOD, DO, temperature, and physical relationships for stream flow, velocity, and depth. The facility of examining intermediate solutions at individual stages and the elimination of the need for continuous differential return functions are two of the principle features of the model.

Long-term power system expansion planning by dynamic programming and production cost simulation

Abstract: This paper extends previously published results on the application of a dynamic programming optimization approach to the planning of systems, with special emphasis on the long-term expansion of power systems. Future uncertainties about loads, equipment costs, etc. are considered explicitly in the method described. A practical computational solution to the resulting stochastic optimization is obtained by means of the recently developed open-loop feedback approximation. Further, it is shown how various known approaches of production cost simulation can be used in conjunction with the dynamic programming optimization to accommodate large systems that may be represented with great technical detail.

Optimal Digital Computer Control of Nuclear Reactors

Abstract: The use of a digital computer to achieve closed loop optimal control of a nuclear reactor is presented. A performance index is defined, and dynamic programming is applied to derive an optimal stationary feedback control law. The control law requires that all state variables be available, therefore nonlinear estimation is used to generate the nonmeasurable system state variables.

Optimal highway staging by dynamic programming

Abstract: A practical procedure for defining an optimal sequence of highway improvements over time, for which computer programs are available, is described. An efficient procedure has been developed by modifying the techniques of dynamic programming, a method which will guarantee the definition of an optimal schedule of independent projects. Specifically, computational efficiency is achieved by use of a variable increment, alternative-oriented approach to dynamic programming, similar to implicit enumeration. Examples demonstrate how dynamic programming can determine the most effective combination of projects when a straight benefit cost analysis cannot.

Heuristic Techniques for Solving Large Combinatorial Problems on a Computer

Abstract: Many problems of optimization, search, or decision-making are combinatorial in nature and basically non-numerical. The advent of the modern high-speed digital computer has opened the way for us to solve many of these problems. Although virtually all of these problems are finite, it is well-known that their size grows extremely rapidly and we can usually expect the computer to do by exhaustive search just a few cases larger than what can be done by hand. Intelligent search procedures such as banach and bound (Little, Murty, Sweeney and Karel (1963), Lawles and Wood (1964)), back-track programming (Golomb and Baument (1965), Walker (1960)), linear or dynamic programming (Gomery (1963), Bellman (1962), Held and Karp (1962)), together with isomorphic rejection (Swift, 1960) help to reduce the total number of cases to be considered but more often than not, we are interested in the solution to problems which are still too large for these techniques. Here some sort of heuristics have to be employed, whereby we hope that the solution (in the instance where a solution when found may be readily verified), or a probably solution, or a useful partial or approximate solution may be obtained in reasonable computation time.

A New Decomposition Procedure for Dynamic Programming

Abstract: This paper presents a new decomposition procedure that reduces the highspeed memory requirement and interpolations, associated with the dynamic programming algorithm. It shows that an nth-order system described by k coupled difference or differential equations can be treated as a kth-order system with respect to the high-speed memory requirement and interpolations in the dynamic programming procedure.

Computational Limitations of Dynamic Programming for Warehouse Location

Abstract: In a recent article, Ballou [1] reported on a dynamic programming method for optimally specifying a warehouse location-relocation plan. A normative model was developed to maximize cumulative profits due to warehouse locations over a time horizon during which locational demand could be predicted with some degree of certainty. Ballou very succinctly stressed the importance of dynamic considerations, including possible warehouse relocations and associated moving costs, in planning a distribution system. The dynamic programming solution method for the model was explicitly stated for a special case when shipments are made to various demand points from a single warehouse. The warehouse could possibly be located in five different locations during the planning period of five years. Ballou termed this case a "limited" problem, and states that the dynamic programming technique could be used for "reasonable-sized" problems, although he does not discuss the extension of the solution procedure to "reasonable-sized" problems. This will show that computation of the solution becomes infeasible if the dynamic programming method is extended without alteration to problems where shipments can be made from a reasonable number of warehouses.

A Dual-Mode Routing Algorithm for an Autonomous Roving Vehicle

Abstract: A dual-mode algorithm for routing an unmanned autonomous roving vehicle designed to explore the uncertain terrain of other planets is presented. The algorithm consists of a global mode, which uses dynamic programming and terrain information available from photo reconnaissance data to determine a nominal optimal path, and a local mode, which routes the vehicle around obstacles whose presence, location, and extent are not known in advance. Gaussian probability density functions are used to simulate terrain for examples that illustrate the performance of the algorithm.

Mathematical Optimization Techniques for the Simultaneous Apportionments of Reliability and Maintainability

Abstract: This paper sets forth mathematical means for optimizing the simultaneous apportionments of reliability and maintainability by means of two techniques. Lagrange multipliers and dynamic programming. Since one of the constraint equations involves allocating both variables, this simultaneous apportionment problem differs from the usual two-dimensional dynamic programming problems. Two cost problems are treated the first minimizes cost while satisfying availability requirements under the assumption that an unlimited ceiling exists for the cost, and the second maximizes reliability and maintainability given a fixed budget. Although these problems appear closely related, step functions usually preclude considering them together. The paper also considers two numerical examples, an interesting cost model for maintainability, and using computers with its models.