Showing papers on "Dynamic programming published in 1976"

Dynamic Programming and Optimal Control, Vol. II

Abstract: A major revision of the second volume of a textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization. The second volume is oriented towards mathematical analysis and computation, and treats infinite horizon problems extensively. New features of the 3rd edition are: 1) A major enlargement in size and scope: the length has increased by more than 50%, and most of the old material has been restructured and/or revised. 2) Extensive coverage (more than 100 pages) of recent research on simulation-based approximate dynamic programming (neuro-dynamic programming), which allow the practical application of dynamic programming to large and complex problems. 3) An in-depth development of the average cost problem (more than 100 pages), including a full analysis of multichain problems, and an extensive analysis of infinite-spaces problems. 4) An introduction to infinite state space stochastic shortest path problems. 5) Expansion of the theory and use of contraction mappings in infinite state space problems and in neuro-dynamic programming. 6) A substantive appendix on the mathematical measure-theoretic issues that must be addressed for a rigorous theory of stochastic dynamic programming. Much supplementary material can be found in the book's web page: http://www.athenasc.com/dpbook.html

Algorithms for Scheduling Independent Tasks

Abstract: The following job sequencing problems are studied: (i) single processor job sequencing with deadlines, (ii) job sequencing on m-identical processors to minimize finish time and related problems, (iii) job sequencing on 2-identical processors to minimize weighted mean flow time. Dynamic programming type algorithms are presented to obtain optimal solutions to these problems, and three general techniques are presented to obtain approximate solutions for optimization problems solvable in this way. The techniques are applied to the problems above to obtain polynomial time algorithms that generate “good” approximate solutions.

Operations Research: Principles and Practice

Abstract: The Nature of Operations Research Linear Programming Network Analysis Advanced Topics in Linear Programming Decision Analysis Random Processes Queueing Models Inventory Models Simulation Dynamic Programming Nonlinear Programming Appendices Index

Optimal Code Generation for Expression Trees

Abstract: This paper discusses algorithms which transform expression trees into code for register machines. A necessary and sufficient condition for optimality of such an algorithm is derived, which applies to a broad class of machines. A dynamic programming algorithm is then presented which produces optimal code for any machine in this class; this algorithm runs in time linearly proportional to the size of the input.

An Improved Long-Run Model for Multiple Warehouse Location

Abstract: This paper proposes an improved model for solving the long-run multiple warehouse location problem. The approach used provides a synthesis of a mixed integer programming formulation for the single-period warehouse location model with a dynamic programming procedure for finding the optimal sequence of configurations over multiple periods. We show that only the Rt, best rank order solutions in any single period need be considered as candidates for inclusion in the optimal multi-period solution. Thus the computational feasibility of the dynamic programming procedure is enhanced by restricting the state space to these Rt best solutions. Computational results on the ranking procedure are presented, and a problem involving two plants, five warehouses, 15 customer zones, and five periods is solved to illustrate the application of the method.

A Solution of the Rectangular Cutting-Stock Problem

Abstract: A method of solving a version of the two-dimensional cutting-stock problem is presented. In this version of the problem one is given a number of rectangular sheets and an order for a specified number of each of certain types of rectangular shapes. The goal is to cut the shapes out of the sheets in such a way as to minimize the waste. However, in many practical applications computation time is also an important economic consideration. For such applications the goal may be to obtain the best solution possible without using excessive amounts of computation time. The method of solution described here avoids exhaustive search procedures by employing an approach utilizing a constrained dynamic programming algorithm to lay out groups of rectangles called strips. This paper also describes the results obtained when the algorithm was tested with some typical rectangular layout problems.

A Preference Order Dynamic Program for Stochastic Assembly Line Balancing

Abstract: Consider an assembly line balancing problem with stochastic task times Our objective is to find a grouping of tasks into stations that satisfies all precedence relations and minimizes the number of stations, subject to the constraint that the probability that the resulting station work content at each station is no more than the given cycle time is bounded by a given value Similar to Held and Karp's approach, we formulate the problem in dynamic programming The solution procedure is based on Mitten's preference order dynamic programming

An Algorithm for Nonlinear Knapsack Problems

Abstract: An algorithm which recursively generates the complete family of undominated feasible solutions to separable nonlinear multidimensional knapsack problems is developed by exploiting discontinuity preserving properties of the maximal convolution. The “curse of dimensionality,” which is usually associated with dynamic programming algorithms, is successfully mitigated by reducing an M-dimensional dynamic program to a 1-dimensional dynamic program through the use of the imbedded state space approach. Computational experience with the algorithm on problems with as many as 10 state variables is also reported and several interesting extensions are discussed.

The synthetic approach to decision table conversion

Abstract: Previous approaches to the problem of automatically converting decision tables to computer programs have been based on decomposition At any stage, one condition is selected for testing, and two smaller problems (decision tables with one less condition) are created An optimal program (with respect to average execution time or storage space, for example) is located only through implicit enumeration of all possible decision trees using a technique such as branch-and-bound The new approach described in this paper uses dynamic programming to synthesize an optimal decision tree from which a program can be created Using this approach, the efficiency of creating an optimal program is increased substantially, permitting generation of optimal programs for decision tables with as many as ten to twelve conditions

Measurable Selection and Dynamic Programming

Abstract: A general multistage problem of stochastic optimization is studied. It is proved under some simple assumptions that the extremum in the problem is attained. The “Bellman functions” are constructed and a criterion of optimality in terms of these functions is given. The main tools used are measurable selection theorems. The paper generalizes the previous work of R. T. Rockafellar and R. J.-B. Wets devoted to the convex case.

A variational approach to optimal route location

Abstract: The paper describes two computer programs which have been developed to optimise the total cost of constructing and using a road. The one depends upon the method of dynamic programming while the second is based upon the method of the variational calculus. An example of their use is reported. /TRRL/

Dynamic file assignment in a computer network

Abstract: We present the models describing a problem of dynamic file allocation in a network of interacting computers and use dynamic programming to solve for the optimal strategies. The problem is treated for the case when the rates of the file requests are known in advance, as well as when only prior statistics is available for these rates. A separation theorem between estimation of the rates and the optimal control is proven, similar to Wonham's separation theorem for linear Gaussian models.

The apparent conflict between estimation and control—a survey of the two-armed bandit problem

Abstract: The two-armed bandit is one of the simplest possible non-deterministic control environments which are not trivial. And yet it is astonishingly difficult to control. For the finite-time problem, dynamic programming methods provide optimal controllers. Optimal control strategies also exist for the infinite-time problem, but their implementation requires infinite storage. Storage can be restricted by allowing only the results of the last r tosses to be recorded: the finite-memory problem—or by considering finite state controllers: the finite-state problem. This paper surveys approaches to the two-armed bandit problem. After introducing the problem and discussing examples of systems where it appears, progress on the three above-mentioned variants is reviewed in turn.

Note---One-Dimensional Facility Location-Allocation using Dynamic Programming

Abstract: A dynamic programming algorithm is presented to locate m variable facilities in relation to n existing facilities situated on one route. Each of the n existing facilities has a requirement flow which must be supplied by the variable facilities. The algorithm does the allocation simultaneously with the location. Computational experience indicates that relatively large problems may be solved optimally.

Multilevel incremental dynamic programing

Abstract: An approach to the solution of multidimensional dynamic programs is presented which differs from most decomposition methods currently used in the optimization of water resources systems. The basic concept is directly dependent on certain characteristics of incremental dynamic programing (IDP). The method, referred to as multilevel incremental dynamic programing, results in a marked reduction in computing time and thus increases the power of IDP as a tool in the optimization of multidimensional deterministic systems.

Markov decision chains with unbounded costs and applications to the control of queues

Abstract: A discrete-time Markov decision model with a denumerable set of states and unbounded costs is considered. It is shown that the optimality equation of dynamic programming along with some additional, easily checked, conditions may be used to establish the optimality or e-optimality of policies with respect to the average expected cost criterion. The results are used to derive optimal policies in two queueing examples. MARKOV DECISION CHAIN; CONTROLLED QUEUE; MINIMUM AVERAGE EXPECTED COST 1. Optimal stationary policies for Markov decision chains 1.

Dynamic programming on recursive reward systems

Abstract: Dynamic programming (DP) has been introduced by R. Bellman [2] as an important technique to solve non-linear programming problems in which a sequence of decisions has to be chosen in an optimal manner. Bellman, in his book, proposed "Principle of Optimality" to show that th e determination of an optimal policy can be reduced to the solution of an optimality equation, i. e., a functional equation that should be satisfied by an optimal return. Although Principle of Optimality is a proposition which needs mathematical reasoning, his justification for the principle was not in a precise mathematical form. For this reason, the scope of cost structure to which the principle is applicable has been left unexplained. Afterward, G. L. Nemhauser [9] gave a sufficient condition for the cost structure in order that an optimality equation holds true. His condition is that the cost function should have both a separability property and a monotonicity property. Nemhauser did not make explicit the relation between the effectiveness of Bellman's principle and the justification for an optimality equation — the relation is no more trivial under his condition. In this paper we shall be concerned with the optimization of finite-stage sequential decision processes. We shall give rigorous proofs for the justification of optimality equations and for the effectiveness of optimality principles with two meanings , without assuming the existence of maximum values of returns. Our condition is that the cost function should have a recursiveness property, a monotonicity property and a Lipschitz condition. Our recursiveness is essentially same as the separability in Nemhauser sense. Our monotonicity has two senses : one is a wide sense, and the other a strict sense. The monotonicity properties in the wide and the strict senses, together with the recursiveness and the Lipschitz condition, induce optimality principles in a weak and a strong senses, respectively. Bellman's Principle of Optimality is well to be identified, in our terms, a principle in the strong sense. Our principle in the weak sense has not been introduced in other literatures as far as the authors know. If we assume the existence of maximum values of returns like Nemhauser did, then the Lipschitz condition can be suppressed from hypotheses in our arguments. In this paper we treat both deterministic and stochastic cases. Section 2 is

Optimal Redundancy Allocation for Non Series-Parallel Networks

Abstract: This paper presents general algorithms for solving redundancy optimization problem for non series-parallel networks. Using closed form expressions for star-delta and delta-star conversions, the exact system reliability expression (objective function) is written down in a straightforward manner. The Box method has been employed to get the optimal continuous solution. The integer solution is obtained by a modified Box method and Branch and bound technique. The scheme is illustrated with an example. The parametric method is used throughout

A practical monthly optimum operations model

Abstract: A mathematical model is developed for the Central Valley Project. The model produces a 12-month operating policy. The model is programmed in FORTRAN IV for processing on a CDC CYBER 74 computer. Project operations managers use the model as a decision-advising tool. This model will be coupled with a daily strategy mode. A four-dimensional incremental dynamic programming algorithm is developed to optimize the operation of the four-reservoir system. The objective of the model is to maximize energy generation within the limits of firm water and power contracts, mandatory releases, and flood control. A unique technique given the acronym RecIP for recomputed initial policy is developed to compute the initial operating policy that is optimized. The RecIP technique is an iterative process using the results of the incremental dynamic programming algorithm to compute a new initial policy to satisfy the required end of operational period target storages desired by the user.

Sigop ii: a new computer program for calculating optimal signal timing patterns

Abstract: This paper describes a new signal timing optimization program, SIGOP II. The optimization procedure consists of two major components: a flow model and an optimization methodology. The objective function of system disutility is expressed directly in terms of vehicle delay, stops, and excess queue length (a congestion deterrent). The flow model computes these components of disutility in the course of the optimization procedure. The optimization procedure uses the method of successive approximations within the framework of a dynamic programming methodology. Gradient techniques are applied to explore a response surface representing system disutility and to locate the minimum value. Associated with this minimum value of disutility is the optimal signal setting sought by the optimization procedure. The platoon structure of traffic and its interaction with the control at the intersection are described. Continuity of flow is preserved from one link to the next. Turning movements, lane channelization, and multiphase control are explicity treated. Practical considerations such as signal split constraints, platoon dispersion, and the effect of short-term fluctuations in volume are included. This program, coded in FORTRAN, is currently being refined and extended. A comparison of SIGOP I and SIGOP II is given.

A planning model for multiple-mode transportation system operations

Abstract: The problem of generating a set of "good" transportation alternatives during the early and intermediate stages of transportation planning is addressed in this paper. A linear programming model of a multi-modal transportation system is developed. The model is run interactively to determine optimal operating levels for all modes for various transport policy decisions. The model described is a component of a composite network generation model incorporating dynamic changes. The linear programming component determines optimal operating policies for given points in time. The composite model incorporates these in a dynamic programming framework to determine optimal staged investment policies over several time periods. /Author/TRRL/

Optimization of design capacity of an aqueduct

Abstract: A methodology to determine aqueduct design capacity under varying damand conditions has been developed. Two models of increasing complexity have been presented. In the first model the demand functions have simultaneous peaks, whereas in the second model they are completely general. Both models are deterministic and use dynamic programming to maximize net benefits. The models have been tested on a hypothetical system.

Curriculum Design Through Operations Research

Abstract: Operations Research techniques have been applied to a variety of educational problems, but rarely, if ever, to the most central decision of the school: what to teach, at what level to teach it, and what to leave out. The necessary formal tools appear now available: The required values may be calculated by the use of the “token strategy” or other judgmental technique. The required time costs may be independently estimated from analysis of materials, or prior class history, or similar aid. Of available OR techniques, the recursive method called dynamic programming is well-suited for optimizing curricular content.

Optimum Structural Design by Dynamic Programming

Abstract: Dynamic programming or recursive optimization is especially useful for multistage processes in which decisions taken at one stage do not effect the previous stages. Dynamic programming is ideally suited for sequential decision processes, such as structural design, involving variables changing at discrete, irregular increments. The theory and computational details of dynamic programming for designing optimum structures is presented herein. The method is illustrated by applying it to the optimum profile design of an extra high voltage transmission tower. The nodal positions, arrangement and type of bracings, and leading dimensions are all treated as variables subject to appropriate constraints. The result is the optimum shape of the tower and arrangement of members. The method can be used for multistory buildings.

Strongly convergent dynamic programming : some results

Abstract: • A submitted manuscript is the author's version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers.

Optimal control with multiple control periods

Abstract: The multiperiod optimal control problem is formulated and solved using dynamic programming.