Showing papers on "Dynamic programming published in 1985"

A Branch and Bound Algorithm for the Total Weighted Tardiness Problem

Abstract: This paper presents a new branch and bound algorithm for the single machine total weighted tardiness problem. It obtains lower bounds using a Lagrangian relaxation approach with subproblems that are total weighted completion time problems. The well-known subgradient optimization technique is replaced by a multiplier adjustment method that leads to an extremely fast bound calculation. The method incorporates various devices for checking dynamic programming dominance in the search tree. Extensive computational results for problems with up to 50 jobs show the superiority of the algorithm over existing methods.

Extensions of the multiarmed bandit problem: The discounted case

Abstract: There are N independent machines. Machine i is described by a sequence {X^{i}(s), F^{i}(s)} where X^{i}(s) is the immediate reward and F^{i}(s) is the information available before i is operated for the sth lime. At each time one operates exacfiy one machine; idle machines remain frozen. The problem is to schedule the operation of the machines so as to maximize the expected total discounted sequence of rewards. An elementary proof shows that to each machine is associated an index, and the optimal policy operates the machine with the largest current index. When the machines are completely observed Markov chains, this coincides with the well-known Gittins index rule, and new algorithms are given for calculating the index. A reformulation of the bandit problem yields the tax problem, which includes, as a special case, Klimov's waiting time problem. Using the concept of superprocess, an index rule is derived for the case where new machines arrive randomly. Finally, continuous time versions of these problems are considered for both preemptive and nonpreemptive disciplines.

Algorithms for Optimizing Hydropower System Operation

Abstract: Successive linear programming, an optimal control algorithm, and a combination of linear program- ming and dynamic programming (LP-DP) are employed to optimize the operation of multireservoir hydrosystems given a deterministic inflow forecast. The algorithm maximize the value of energy pro- duced at on-peak and off-peak rates, plus the estimated value of water remaining in storage at the end of the 12-month planning period. The LP-DP algorithm is clearly dominated: it takes longer to find a solution and produces significantly less hydropower than the other two procedures. Successive linear programming (SLP) appears to find the global maximum and is easily implemented. For simple systems the optimal control algorithm finds the optimum in about one fifth the time required by SLP but is harder to implement. Computing costs for a two-reservoir, 12-month deterministic problem averaged about seven cents per run using optimal control and 37 cents using successive linear programming.

The Joint Replenishment Problem with a Powers-of-Two Restriction

Abstract: In this paper we consider the joint replenishment problem in the light of recent work by the second and third authors concerning the selection of realistic and consistent reorder intervals in production/ distribution systems. After stating a general dynamic programming formulation of the joint replenishment problem, we present its usual statement which assumes constant reorder intervals. We then restrict the problem further by assuming the constant reorder intervals are powers-of-two multiples of some base planning interval. We present an algorithm based on concepts we developed that solves the joint replenishment problem with the powers-of-two restriction. Like other algorithms proposed for this problem, it is a simple sorting algorithm. Finally, we establish that the algorithm yields a solution whose average annual cost is within 6% of the general problem's long-run minimum average annual cost.

Optimal Policy for a Two-Stage Assembly System under Random Demand

Abstract: This paper considers an inventory system in which an end product is assembled from two components, each of which is ordered from an external supplier. Only the end product has final demand, which is assumed to be random. Using the functional equation approach of dynamic programming, we characterize the forms of both the optimal order policy for the components and the optimal assembly policy of the end product for a multiperiod problem of arbitrary, but finite length. We believe that this work represents the only exact analysis of an MRP-type assembly system when demand is stochastic.

Additive control of stochastic linear systems with finite horizon

Abstract: We consider a dynamic system whose state is governed by a linear stochastic differential equation with time-dependent coefficients. The control acts additively on the state of the system. Our objective is to minimize an integral cost which depends upon the evolution of the state and the total variation of the control process. It is proved that the optimal cost is the unique solution of an appropriate free boundary problem in a space-time domain. By using some decomposition arguments, the problems of a two-sided control, i.e. optimal corrections, and the case with constraints on the resources, i.e. finite fuel, can be reduced to a simpler case of only one-sided control, i.e. a monotone follower. These results are applied to solving some examples by the so-called method of similarity solutions.

Optimal control of jump-linear gaussian systems†

Abstract: This paper investigates the problem of controlling a discrete-time linear system with jump parameters. A review of the literature is presented as well as a development of the application of dynamic programming to this class of control problems. Dynamic programming has been applied by many researchers and it was observed that no closed-form analytical solution could be constructed because of the ‘dual’ aspects of the controller. The main contribution of the present work is an algorithm, suitable for computer implementation, for the optimal dual control. The construction of the algorithm is based on transforming the dynamic programming relations into a space of sufficient statistics and using a finite-dimensional optimization procedure to obtain the optimal control as a function of the statistics. This is achieved by first developing a suitable recursive realization of a ‘filter’ which generates the sufficient statistics for the problem and then embedding this filter into the dynamic programming equations. ...

A dynamic programming algorithm to find all solutions in a neighborhood of the optimum

Abstract: Just after he introduced dynamic programming, Richard Bellman with R. Kalaba in 1960 gave a method for finding Kth best policies. Their method has been modified since then, but it is still not practical for many problems. This paper describes a new technique which modifies the usual backtracking procedure and lists all near-optimal policies. This practical algorithm is very much in the spirit of the original formulation of dynamic programming. An application to matching biological sequences is given.

A Solution of the Unit Commitment Problem Via Decomposition and Dynamic Programming

Abstract: Each day power generating units have to be selected to realize a reliable production of electric energy with the fewest fuel costs. This paper proposes decomposition and dynamic programming as techniques for solving the unit commitment problem, a high- dimensional non-linear, mixed-integer optimization problem. Experiments indicate that the proposed methods locate in less time a better solution than many existing techniques.

REDUCING THE PARETO OPTIMAL SET IN MULTICRITERIA OPTIMIZATION(With Applications to Pareto Optimal Dynamic Programming)

Abstract: This paper describes the rationale for using clustering methods to reduce the size of the Pareto optimal set whilst retaining its shape. It proceeds lo describe an implementation of the complete-linkage clustering method and demonstrates its application. Finally, the method is incorporated into a Pareto optimal serial dynamic programming process to reduce the size of the Pareto optimal set generated at each stage of the optimization.

Minimizing setups in ordered sets of fixed width

Abstract: A simply polynomial time algorithm is given for computing the setup number, or jump number, of an ordered set with fixed width. This arises as an interesting application of a polynomial time algorithm for solving a more general weighted problem in precedence constrained scheduling.

Stereo by two-level dynamic programming

Abstract: This paper presents a stereo algorithm using dynamic programming technique. The stereo matching problem, that is, obtaining a correspondence between right and left images, can be cast as a search problem. When a pair of stereo images is rectified, pairs of corresponding points can be searched for within the same scanlines. We call this search intra-scanline search. This intra-scanline search can be treated as the problem of finding a matching path on a two dimensional (2D) search plane whose axes are the right and left scanlines. Vertically connected edges in the images provide consistency constraints across the 2D search planes. Inter-scanline search in a three-dimensional (3D) search space, which is a stack of the 2D search planes, is needed to utilize this constraint. Our stereo matching algorithm uses edge-delimited intervals as elements to be matched, and employs the above mentioned two searches: one is inter-scanline search for possible correspondences of connected edges in right and left images and the other is intra-scanline search for correspondences of edge-delimited intervals on each scanline pair. Dynamic programming is used for both searches which proceed simultaneously in two levels: the former supplies the consistency constraints to the latter while the latter supplies the matching score to the former. An interval-based similarity metric is used to compute the score.

Pumped Wastewater Collection Systems Optimization

Abstract: Identifying the need for intermediate and/or end pumping in real life wastewater collection systems, an optimization algorithm is developed for the design of gravity‐cum‐pumped systems with recourse to dynamic programming. The problem of dimensionality is minimized through cost effective feasible groupings at junction manholes and with division of algorithm in two parts. The first part identifies optimal control variables associated with each link and stores the same while the second part uses these stored values along with input data to prepare detailed hydraulic and cost statements. The effectiveness of the algorithm is tested through two case studies.

Experiments with parallel algorithms for combinatorial problems

Abstract: In the last decade many models for parallel computation have been proposed and many parallel algorithms have been developed. However, few of these models have been realized and most of these algorithms are supposed to run on idealized, unrealistic parallel machines. The parallel machines constructed so far all use a simple model of parallel computation. Therefore, not every existing parallel machine is equally well suited for each type of algorithm. The adaptation of a certain algorithm to a specific parallel archi- tecture may severely increase the complexity of the algorithm or severely obscure its essence. Little is known about the performance of some standard combinatorial algorithms on existing parallel machines. In this paper we present computational results concerning the solution of knapsack, shortest paths and change-making problems by branch and bound, dynamic programming, and divide and conquer algorithms on the ICL-DAP (an SIMD computer), the Manchester dataflow machine and the CDC-CYBER-205 (a pipeline computer). 1980 Mathematics Subject Classification: 90C27, 68Q10, 68R05. This paper appeared in European Journal of Operational Research, Vol. 33, pp 65-81, 1988.

Optimal Operation of Multireservoir Systems Using an Aggregation-Decomposition Approach

Abstract: A new application of the Aggregation-Decomposition approach (AD) to the optimal scheduling of large hydrothermal generation systems with multiple reservoirs is presented. The problem (with N reservoirs) is decomposed into N subproblems with two state variables. Each subproblem finds the optimal operating policy for one of the reservoir as a function of the energy content of that reservoir and the aggregate energy content of the remaining reservoirs. The subproblems are solved by stochastic dynamic programming taking into account the detailed models of the hydro chains as well as the stochasticity and correlation of the hydro inflows. The method has been successfully implemented on a 10 reservoir hydrothermal power system.

The Principle of Optimality in Dynamic Programming with Returns in Partially Ordered Sets

Abstract: A general sequential model is defined where returns are in a partially ordered set. A distinction is made between maximal (nondominated) returns and greatest returns. We show that Bellman's principle of optimality is valid with respect to maximal returns and it leads to an algorithm to approximate these returns. We argue that significantly greater effort is needed to apply this algorithm to maximal returns than to greatest returns.

Some Results on Bellman Equation in Hilbert Spaces

Abstract: We give an existence result on the Bellman equation related to an infinite dimensional control problem.

A Risk-Constrained Stochastic Dynamic Programming Approach To The Operation Planning Of Hydrothermal Systems

Abstract: This work presents a new approach to tlhee problem of finding opcperating strategies for a hydthermalro power generating system. The objective is to minimize the expected thermal generating costs subject to prbabilistic constraints on the failure to supply the energy load. The problem is solved by a stochastic dynamic programing algorithm with nested reliability constraints fromcn each stage to the end of the planning period. A decomacposition approach is used to extend the methodology to the operation of two interconnmected systems. Case studies with the South and Southeast Brazilian generating systems are presented and discussed.

Note—A Dynamic Programming Algorithm for Joint Replenishment Under General Order Cost Functions

Abstract: The multi-item joint replenishment problem is generalized to allow ordering costs to be dependent on the specific items jointly supplied. A fixed cycle approach is examined in which all the items of a group are always jointly replenished. A dynamic programming algorithm is developed for partitioning the items into groups, each with its own fixed cycle time, resulting in an optimal fixed cycle replenishment policy.

Application of linear and dynamic programming to the optimization of the production of hydroelectric power

Abstract: Application of the methods of control theory to industrial problems demands a match of the necessary computational tools to the actual financial constraints. The development of suboptimal control algorithms allows performance gains that were limited in the past to installations with extensive computer facilities to be reached now by smaller ones. Advances in this domain lead naturally to a rational use of modern control tools based on mini- and microcomputers. Within this framework, we present a general method for the optimal control of electric power plants. The optimization problem is described and formulated as the optimal control of a multivariable state-space model in which the state and control vectors are constrained by sets of equality or inequality relations. The solution is obtained in two steps with linear and dynamic programming methods; the results are expressed in the form of parametric algorithms which set up the working point of the turbine-generator units so that the resulting profit represents a maximum. The application of the method to the optimization of the production of a Swiss electricity company illustrates the approach.

Dynamic model for scheduling maintenance of transportation facilities

Abstract: The optimum scheduling of maintenance for transportation facilities is addressed. The problem is described as a multiple-period resource allocation problem with constraints on both resource availability and state and decision variables. The problem is formulated as a nonlinear optimization problem and solved by using the generalized reduced gradient method. The model uses recursive formulas similar to those used in dynamic programming in order to calculate the partial derivatives of the objective function. The model is applied to an example based, in part, on actual data provided by the Japanese National Railways. Several tests are made to show the performance of the model, and the results are compared with those of two alternative solutions. The results show the usefulness of the model in a wide variety of applications and its superiority to the alternative solutions examined.

Vector Optimal Routing by Dynamic Programming

Abstract: Algorithms for generating shortest paths have been widely studied in the combinational optimisation literature for many years. More recently, the vector version, in which non-dominated paths are sought, has been investigated. Acyclic problems were discussed by Randolph and Ringeisen1 and Thuente.2 The bicriterion case was analysed by Hansen3 and extended by Climaco and Martins4 and Martins.5 A multicriterion algorithm was given by Martins.6 White7 used linear programming methodology but was forced to exclude non-dominated paths which were dominated by a convex combination of path lengths.

Technical Note—Forest Outturn Optimization by Dantzig-Wolfe Decomposition and Dynamic Programming Column Generation

Abstract: In this paper, we show how Dantzig-Wolfe decomposition, with delayed column generation by means of dynamic programming, can be used to determine the optimal set of bucking policies for entire stands of trees. Determined in this way, the bucking policies are consistent with wood resource qualities and quantities and end use product requirements, and represent the activities of a linear programming model that covers supply and demand constraints. We determine optimal bucking policies using dynamic programming techniques.

Asymptotic Evolution of a Stochastic Control Problem

Abstract: Here we study the minimization problem of an integral discounted functional, on a set of nonexplosive and nonconstrained diffusions. The integrand is “weakly coercive,” which leads us, using the dynamic programming method, to characterize the optimal cost among the solutions of the solving equation, with radiative conditions expressing the centripetal aspect of the optimal control. The evolution of the problem when the discount vanishes is then considered: the integrand being “strongly coercive” (i.e. its gradient being outward), a limit problem is defined and similarly solved; an inward optimal control exists, which is the limit of the ones of the initial problems. The existence properties are obtained by means of a priori estimates concerning suitable solutions of the solving equations in the whole space.