Showing papers on "Dynamic programming published in 1968"
TL;DR: The detailed allocation method of system reliability discussed in this paper is designed to select the optimal solution in the context of the trade-off analysis.
Abstract: The detailed allocation method of system reliability discussed in this paper is designed to select the optimal solution in the context of the trade-off analysis. It is noted that the problem may be structured as an n-stage sequential decision problem. A computational algorithm is developed using dynamic programming.
343 citations
TL;DR: In this article, the application of dynamic programming techniques to solve optimization problems that occur in the short-term (transient) terminal distribution and longterm (steady-state) transmission of natural gas is summarized.
Abstract: The complexity and expense of operating natural-gas pipeline systems have made optimum operation and planning of increased interest to the natural-gas pipeline industries. Since the operations of natural-gas pipeline sytems are characterized by inherent nonlinearities and numerous constraints, dynamic programming provides an extremely powerful method for optimizing such systems. This paper summarizes the application of dynamic programming techniques to solve optimization problems that occur in the short-term (transient) terminal distribution and long-term (steady-state) transmission of natural gas.
183 citations
TL;DR: In this article, two methods for computing optimal decision sequences and their cost functions are presented for solving a broad class of shortest-route problems and a third solution technique is shown to apply to certain, but not all, of these Markov renewal programs.
Abstract: : Two methods are presented for computing optimal decision sequences and their cost functions. The first method, called 'policy iteration,' is an adaption of an iterative scheme that is widely used for sequential decision problems. The second method is to specify a linear programming problem whose solution determines an optimal policy and its cost function. A third solution technique is shown to apply to certain, but not all, of these Markov renewal programs. As a byproduct of the development, new techniques are provided for solving a broad class of shortest-route problems. (Author)
153 citations
TL;DR: The notion of differential dynamic programming is used to develop new second-order and first-order successive-approximation methods for determining optimal control and these methods have been extended to solve problems with terminal constraints and implicitly given final time.
Abstract: In this paper, the notion of differential dynamic programming is used to develop new second-order and first-order successive-approximation methods for determining optimal control.
98 citations
TL;DR: Certain properties of the optimal path are deduced from the dynamic programming formulation of the shortest route problem, and these properties are utilized in the construction of an algorithm that finds the optimal route.
Abstract: Given a finite horizon delivery schedule for n products we wish to schedule production on a single machine to meet deliveries and minimize the number of change-overs of the machine from one product to another. A state space is defined and in it a network is constructed such that the shortest distance through the network corresponds to the minimum number of production change-overs. Certain properties of the optimal path are deduced from the dynamic programming formulation of the shortest route problem, and these properties are utilized in the construction of an algorithm that finds the optimal path. A numerical example illustrates the method.
64 citations
TL;DR: Differential dynamic programming as discussed by the authors is a technique, based on dynamic programming rather than the calculus of variations, for determining the optimal control function of a nonlinear system, which applies the principle of optimality in the neighborhood of a nominal, possibly nonoptimal, trajectory.
Abstract: Differential dynamic programming is a technique, based on dynamic programming rather than the calculus of variations, for determining the optimal control function of a nonlinear system. Unlike conventional dynamic programming where the optimal cost function is considered globally, differential dynamic programming applies the principle of optimality in the neighborhood of a nominal, possibly nonoptimal, trajectory. This allows the coefficients of a linear or quadratic expansion of the cost function to be computed in reverse time along the trajectory: these coefficients may then be used to yield a new improved trajectory (i.e., the algorithms are of the "successive sweep" type). A class of nonlinear control problems, linear in the control variables, is studied using differential dynamic programming. It is shown that for the free-end-point problem, the first partial derivatives of the optimal cost function are continuous throughout the state space, and the second partial derivatives experience jumps at switch points of the control function. A control problem that has an aualytic solution is used to illustrate these points. The fixed-end-point problem is converted into an equivalent free-end-point problem by adjoining the end-point constraints to the cost functional using Lagrange multipliers: a useful interpretation for Pontryagin's adjoint variables for this type of problem emerges from this treatment. The above results are used to devise new second- and first-order algorithms for determining the optimal bang-bang control by successively improving a nominal guessed control function. The usefulness of the proposed algorithms is illustrated by the computation of a number of control problem examples.
58 citations
TL;DR: In the first section a modification to Howard's policy improvement routine for Markov decision problems is described and a particular form of recurrence relation is used to show that the repair limit method is the optimal strategy for a basic equipment replacement problem.
Abstract: In the first section a modification to Howard's policy improvement routine for Markov decision problems is described. The modified routine normally converges the more rapidly to the optimal policy. In the second section a particular form of recurrence relation, which leads to the rapid determination of improved policies is developed for a certain type of dynamic programming problem. The relation is used to show that the repair limit method is the optimal strategy for a basic equipment replacement problem.
46 citations
19 citations
TL;DR: A decomposition procedure for extending the size of problems that can be solved using dynamic programming by decomposing the tabular arrays of data into blocks of data and performing the dynamic programming calculations over the whole tabular array by calculating on each block separately.
Abstract: This paper presents a decomposition procedure for extending the size of problems that can be solved using dynamic programming It essentially consists of decomposing the tabular arrays of data into blocks of data, and then performing the dynamic programming calculations over the whole tabular array by calculating on each block separately
19 citations
TL;DR: A method is presented for selecting a subset of features from a specified set when economic considerations prevent utilization of the complete set, and the formulation of the feature selection problem as a dynamic programming problem permits an optimal solution to feature selection problems which previously were uncomputable.
Abstract: A method is presented for selecting a subset of features from a specified set when economic considerations prevent utilization of the complete set. The formulation of the feature selection problem as a dynamic programming problem permits an optimal solution to feature selection problems which previously were uncomputable. Although optimality is defined in terms of a particular measure, the Fisher return function, other criteria may be substituted as appropriate to the problem at hand. This mathematical model permits the study of interactions among processing time, cost, and probability of correctly classifying patterns, thus illustrating the advantages of dynamic programming. The natural limitation of the model is that the only features which can be selected are those supplied by its designer. Conceptually, the dynamic programming approach can be extended to problems in which several constraints limit the selection of features, but the computational difficulties become dominant as the number of constraints grows beyond two or three.
19 citations
TL;DR: In this paper, the authors apply dynamic programming to the problem of locating the optimal stress distribution, and then the optimal design is obtained, where realistic cost functions are introduced without difficulty, taking account of nonlinearity, fabrication cost and limited section availability.
Abstract: The lower bound theorem of plastic theory offers a direct procedure for structural design: any equilibrium distribution of stress (or force, or bending moment) gives an admissible design if the structure is so proportioned that it is then everywhere at or below yield. A cost can be assigned to each element of the structure, and the total cost is the sum of these costs. The optimal structure is the cheapest of all admissible designs, and is associated with the optimal equilibrium stress distribution. Herein dynamic programming is applied to the problem of locating this optimal stress distribution, and thence the optimal design. An application to the optimal design of a continuous beam is described in detail. It turns out that realistic cost functions can be introduced without difficulty, taking account of nonlinearity, fabrication cost and limited section availability. Extensions of the technique to more complex structures are discussed, and illustrated by the design of a multistory single-bay frame.
01 Jun 1968
TL;DR: In this paper, an efficient method is given for finding minimal cost-time ratio circuits in routing problems through the use of the out-of-kilter algorithm, and the main difference between earlier approaches and the present one is that where others have used a shortest path method corresponding to the simplex method, except that steepest descent is not used.
Abstract: : An efficient method is given for finding minimal cost-time ratio circuits in routing problems through the use of the out-of-kilter algorithm. The problem of finding a cycle in a network having a minimal cost-to-time ratio has been considered previously, and column generators have been used to introduce, into the basis of the master problem, the solution that corresponds to this cycle. The subproblem is of independent interest and corresponds to deterministic single chain Markov renewal programming. The main difference between earlier approaches and the present one is that where others have used a shortest path method corresponding to the simplex method (except that steepest descent is not used), here the flow circulation problem for optimal cost-time tradeoff is solved parametrically by the out-of-kilter algorithm. (Author)
01 Nov 1968
TL;DR: An investigation of five algorithms for finding the shortest path from a root node to all other nodes for different network structures found that one is best suited for their problem.
Abstract: : Efficiency of shortest path algorithms is a function of various network parameters. This paper reports the results of an investigation of five algorithms for finding the shortest path from a root node to all other nodes for different network structures. Parameters considered are number of nodes, number of links, range of data (i.e., arc lengths), and shape (if applicable). It is hoped that the results will suggest to potential users of these algorithms which one is best suited for their problem.
TL;DR: This correspondence describes the formulation and solution of a nonlinear, non-Gaussian stochastic control problem, used to obtain the solution to the problem of optimally controlling a robot, equipped with sensors, that is operating in an unknown environment.
Abstract: This correspondence describes the formulation and solution of a nonlinear, non-Gaussian stochastic control problem. Dynamic programming is used to obtain the solution to the problem of optimally controlling a robot, equipped with sensors, that is operating in an unknown environment.
TL;DR: The technique of dynamic programming is presented as a method of optimizing the staging sequence of urban highway improvements by determining the optimal sequence of highway improvement from the many sequences that are possible.
Abstract: The technique of dynamic programming is presented as a method of optimizing the staging sequence of urban highway improvements. The technique provides a method of determining the optimal sequence of highway improvement from the many sequences that are possible. An example urban freeway system is used to demonstrate the technique of dynamic programming. The optimal sequence of staging decisions for two decision sets are determined.
TL;DR: The structuring of algorithms suitable for execution on parallel processors is discussed and a restructuring of Bellman's dynamic programming technique is given.
Abstract: The structuring of algorithms suitable for execution on parallel processors is discussed. Two examples of such algorithms are given. The first example exhibits a restructuring of Bellman's dynamic programming technique; the second presents a method of parsing MAD-type statements in parallel.
TL;DR: If a dynamic program has uncertainties of linear form in the constraint or objective function, a problem may be formulated to take this into account, and this new problem will be solvable by dynamic programming with a larger state space.
Abstract: If a dynamic program has uncertainties of linear form in the constraint or objective function, a problem may be formulated to take this into account This new problem, under certain assumptions, wi
TL;DR: The present paper states mathematically and solves the problem of an optimal search method for one class of functions and defines the optimal search algorithm for the case when the function is computed with known errors (and in particular, accurately).
Abstract: IN a wide range of practical and computational problems we have to find the roots of a function which can only be computed approximately for given values of its argument. Examples include the boundary value problem for a nonlinear system of differential equations, solved by specifying missing initial conditions (at one end), or the problem of the experimental selection of the parameters of a process or device in order to satisfy a given condition. Since computation of the function whose root is required often involves a laborious, lengthy or expensive procedure, it seems natural to consider optimization of the method of finding the root. Two problems arise here: first, assuming that the errors in computing the function are known, how to select the points at which it is to be computed; and second, how optimally to distribute the existing resources over the computational stages. By resources we mean what the computational accuracy depends on in a specific case, e.g. computer time, labour or cost of the computations, etc. The present paper states mathematically and solves the problem of an optimal search method for one class of functions. The statement comes in Section 1, while Section 2 defines the optimal search algorithm for the case when the function is computed with known errors (and in particular, accurately). The question of the optimal distribution of the resources over the computations is discussed in Section 3. Some examples are considered, and the results of calculating optimal algorithms are given. In the course of the solution we use ideas of the method of dynamic programming, which were earlier used for devising opt1imal search algorithms in which computational errors were ignored [1].
TL;DR: This note shows that the linear programming formulation and his extension based on it are defective, and that the dynamic programming algorithm can lead to suboptimal solutions, but a minor change in theynamic programming formulation relieves the difficulty.
Abstract: Romesh Saigal presents a zero-one linear program and a dynamic programming algorithm for finding the shortest route containing exactly q arcs from node 1 to node n in a network N, A with distances ci,j. This note shows that the linear programming formulation and his extension based on it are defective, and that the dynamic programming algorithm can lead to suboptimal solutions, but a minor change in the dynamic programming formulation relieves the difficulty.
TL;DR: The linear programming solution to Markov chain theory models is presented and compared to the dynamic programming solution and it is shown that the elements of the simplex tableau contain information relevant to the understanding of the programmed system.
Abstract: Some essential elements of the Markov chain theory are reviewed, along with programming of economic models which incorporate Markovian matrices and whose objective function is the maximization of the present value of an infinite stream of income. The linear programming solution to these models is presented and compared to the dynamic programming solution. Several properties of the solution are analyzed and it is shown that the elements of the simplex tableau contain information relevant to the understanding of the programmed system. It is also shown that the model can be extended to cover, among other elements, multiprocess enterprises and the realistic cases of programming in the face of probable deterioration of the productive capacity of the system or its total destruction.
TL;DR: The formulation of a firm growth problem in a dynamic programming framework is illustrated and the type of results obtained is contrasted with that normally obtained from dynamic linear programming formulations.
Abstract: This paper is concerned with dynamic programming as a tool for studying the process of farm firm growth. Studies of growth are restricted by characteristics of the analytical tools used. Dynamic programming provides a method for including added realism in conceptual and analytical growth models. This paper illustrates the formulation of a firm growth problem in a dynamic programming framework and discusses advantages and disadvantages of such models. The type of results obtained from dynamic programming is contrasted with that normally obtained from dynamic linear programming formulations.
TL;DR: An iterative dynamic programming procedure for the minimization of the annual operating cost is described and a continuous formulation which represents the limiting design for an infinite number of stages is presented.
TL;DR: An LP is formulated to which the decomposition principle is applied and a nonlinear programming problem in integer variables is generated as a subproblem to be generated at each iteration by dynamic programming approach.
Abstract: Given a set {di} of continuous rates of demand for n products, the associated costs, unit processing times, and set-up times, and given N different types of processors with possibly more than one processor per type, it is desired to determine the single-stage production pattern under the constraint that only EMQ's are manufactured. An LP is formulated to which we apply the decomposition principle and generate a nonlinear programming problem in integer variables as a subproblem to be generated at each iteration. This subproblem is solved by dynamic programming approach.
01 Jan 1968
TL;DR: Attention is drawn to a new class of non-linear integer programming problems which arise during the course of the discussion.
Abstract: The technique of dynamic programming is employed to find the number of subevents into which each event of a finite generalised probability scheme should be divided so as to optimize the expressions for entropies given by Shannon, Renyi and the author. Attention is drawn to a new class of non-linear integer programming problems which arise during the course of the discussion.
15 Jul 1968
TL;DR: The aim of this paper is to illustrate the application of backtrack programming to the design of welded plate girders.
Abstract: The object of engineering design is to satisfy some need of man with the maximization or minimization of some measure of effectiveness of the solution. Common measures of effectiveness are cost, cost-benefit ratio, and profit. In mathematical terminology an object or facility can be described by a list or vector of parameter values. The position of each element in the vector associates it with a particular parameter. The performance of the object or facility and the constraints imposed on the performance are described by a set of equalities and inequalities called the design equations. The effectiveness of the solution is indicated by the value obtained by evaluating an objective function with the design parameter values. The object of the design process is to determine the vector of parameter values, the optimum solution, which satisfies the design equations and maximizes or minimizes, as appropriate, the value of the objective function.There are a number of mathematical procedures which are useful in optimization problems. However, most methods are applicable only to particular classes of problems. The application of linear programming, for example, is limited to problems in which the design variables are involved only in linear relationships. Dynamic programming is applicable to problems in which there is a sequential flow of information. Gradient search methods are useful in many problems, but may lead to incorrect results if the function is not unimodal. Difficulties are encountered also if one or more of the parameters are defined only at discrete values. Backtrack programming, 2,3on the other hand, is generally applicable to optimization problems including those for which more specialized techniques are available. The aim of this paper is to illustrate the application of backtrack programming to the design of welded plate girders.
TL;DR: The potential use of multiprocessing computers for possible improvement of dynamic programming solutions is considered and the dimensionality restrictions and the search in case of a multidimensional control vector are discussed.
Abstract: The potential use of multiprocessing computers for possible improvement of dynamic programming solutions is considered. In particular, the dimensionality restrictions and the search in case of a multidimensional control vector are discussed. While the dimension of a practically solvable problem would be increased only slightly, a considerable improvement could be expected in case of a parallel search for a multidimensional control vector.