Showing papers on "Dynamic programming published in 1988"

Neural networks for nonlinear programming

Abstract: The dynamics of the modified canonical nonlinear programming circuit are studied and how to guarantee the stability of the network's solution. By considering the total cocontent function, the solution of the canonical nonlinear programming circuit is reconciled with the problem being modeled. In addition, it is shown how the circuit can be realized using a neural network, thereby extending the results of D.W. Tank and J.J. Hopefield (ibid., vol.CAS-33, p.533-41, May 1986) to the general nonlinear programming problem. >

The multiple sequence alignment problem in biology

Abstract: The study and comparison of sequences of characters from a finite alphabet is relevant to various areas of science, notably molecular biology. The measurement of sequence similarity involves the consideration of the different possible sequence alignments in order to find an optimal one for which the “distance” between sequences is minimum. By associating a path in a lattice to each alignment, a geometric insight can be brought into the problem of finding an optimal alignment. This problem can then be solved by applying a dynamic programming algorithm. However, the computational effort grows rapidly with the number N of sequences to be compared $(O(l^N ))$, where l is the mean length of the sequences to be compared).It is proved here that knowledge of the measure of an arbitrarily chosen alignment can be used in combination with information from the pairwise alignments to considerably restrict the size of the region of the lattice in consideration. This reduction implies fewer computations and less memory ...

Vehicle routing with time windows: optimization and approximation. vehicle routing: method and studies. studies in management science and systems - volume 16

Abstract: This is a survey of solution methods for routing problems with time window constraints. Among the problems considered are the traveling salesman problem, the vehicle routing problem, the pickup and delivery problem, and the dial-a-ride problem. We present optimization algorithms that use branch and bound, dynamic programming and set partitioning, and approximation algorithms based on construction, iterative improvement and incomplete optimization. (Author/TRRL)

Using Dynamic Programming For Minimizing The Energy Of Active Contours In The Presence Of Hard Constraints

Abstract: Energy-Minimizing Active Contour Models (snakes) have recently been proposed by Kass et al. [8] as a top-down mechanism for locating features of interest in images. The Kass et al.’s algorithm involves four steps: setting up a variational integral on the continuous plane, deriving a pair of Euler equations, discretizing them, and solving the discrete equations iteratively until convergence. This algorithm suffers from a number of problems. We discuss these problems and present an algoIithm for active contours based on dynamic programming. The optimization problem is set up as a discrete multi-stage decision process and is solved by a “time-delayed” discrete dynamic programming algorithm. This formulation leads to a stable behavior for the active contours over iterations, in addition to allowing for hard constraints to be enforced on the behavior of the solution. Results of the application of the proposed algorithm to real images is presented.

Vehicle routing with time windows: Optimization and approximation

Abstract: A survey of solution methods for routing problems with time window constraints. Among the problems considered are the traveling salesman problem, the vehicle routing problem, the pickup and delivery problem, and the dial-a-ride problem. Optimization algorithms that use branch and bound, dynamic programming and set partitioning, and approximation algorithms based on construction, iterative improvement and incomplete optimization are presented.

An enhanced dynamic programming approach for unit commitment

Abstract: To investigate modeling problems with predefined unit hourly start-up ramps, the authors look at the method by which dynamic programming (DP) considers and reject combinations. An enhanced DP approach which saves predecessor options was developed and implemented in an online energy-management system. The method not only supports realistic modelling of unit start-up ramps, but it also involves the analysis of solution paths that may have been eliminated under traditional methods, thereby making better solutions possible. Sample results are given to demonstrate the benefits of the proposed algorithm. >

Short-term scheduling of thermal-electric generators using Lagrangian relaxation

Abstract: This paper presents an expanded formulation of the unit commitment problem in which hundreds of thermal-electric generators must be scheduled on an hourly basis, for up to 7 days at a time. The underlying model incorporates the full set of costs and constraints including setup, production, ramping, and operational status, and takes the form of a mixed integer nonlinear control problem. Lagrangian relaxation is used to disaggregate the model by generator into separate subproblems which are then solved with a nested dynamic program. The strength of the methodology lies partially in its ability to construct good feasible solutions from information provided by the dual. Thus, the need for branch-and-bound is eliminated. In addition, the inclusion of the ramping constraint provides new insight into the limitations of current techniques. Computational experience with the proposed algorithm indicates that problems containing up to 100 units and 48 time periods can be readily solved in reasonable times. Duality gaps of less than 1% were achieved in all cases.

An Adaptable Scheduling Algorithm for Flexible Flow Lines

Abstract: Consider a manufacturing line that produces parts of several types. Each part must be processed by at most one machine in each of several banks of machines. This paper presents an algorithm that schedules the loading of parts into such a line. The objective is primarily to minimize the makespan and secondarily to minimize queueing. The problem is decomposed into three subproblems and each of these is solved using a fast heuristic. The most challenging subproblem is that of finding a good loading sequence, and this is addressed using workload concepts and an approximation to dynamic programming. We make several extensions to the algorithm in order to handle limited storage capacity, expediting, and reactions to system dynamics. The algorithm was tested by computing schedules for a real production line, and the results are discussed.

An optimization method for load management scheduling

Abstract: Many utilities have load management programs whereby they directly control residential appliances in their service area. An algorithm for scheduling the load control using dynamic programming is presented. This method is based on an analytic dynamic model of the load under control. The method can be used for different utility objectives, including minimizing production cost and minimizing peak load over a period of time. >

Dynamic Programming State-Space Relaxation for Single-Machine Scheduling

Abstract: The problem of sequencing jobs on a single machine to minimize total cost is considered. Machine capacity constraints require that, at any time, at most one job is processed. Also, no machine idle-time between processing jobs is allowed. In contrast to most research, it is not assumed that the cost is a non-decreasing function of completion time. A dynamic programming formulation of the problem is presented. Since the number of states required by this formulation is prohibitively large, the possibilities for branch and bound algorithms are explored. It is shown that the dynamic programming formulation can be relaxed by mapping the state-space onto a smaller state-space and performing the recursion on this smaller state-space, thereby giving a lower bound. Techniques for improving this lower bound through the use of penalties and through the use of state-space modifiers are discussed. Computational results are presented for the problem in which each job has a due date, and the objective is to minimize the sum of holding costs for jobs completed before their due date and tardiness costs for jobs completed after their due date.

Using common subexpressions to optimize multiple queries

Abstract: The problem of identifying common subexpressions and using them in the simultaneous optimization of multiple queries is dealt with. In particular, emphasis is placed on the strategy of selecting access plans for single queries and their integration into a global access plan that takes advantage of common tasks. A dynamic programming algorithm is presented for the selection of individual access plans such that the resulting global access plan is of minimum processing cost. The computational complexity of this algorithm represents a significant improvement over existing algorithms. >

Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems

Abstract: A new computational algorithm is presented for the solution of discrete time linearly constrained stochastic optimal control problems decomposable in stages. The algorithm, designated gradient dynamic programming, is a backward moving stagewise optimization. The main innovations over conventional discrete dynamic programming (DDP) are in the functional representation of the cost-to-go function and the solution of the single-stage problem. The cost-to-go function (assumed to be of requisite smoothness) is approximated within each element defined by the discretization scheme by the lowest-order polynomial which preserve its values and the values of its gradient with respect to the state variables at all nodes of the discretization grid. The improved accuracy of this Hermitian interpolation scheme reduces the effect of discretization error and allows the use of coarser grids which reduces the dimensionality of the problem. At each stage, the optimal control is determined on each node of the discretized state space using a constrained Newton-type optimization procedure which has quadratic rate of convergence. The set of constraints which act as equalities is determined from an active set strategy which converges under lenient convexity requirements. This method of solving the single-stage optimization is much more efficient than the conventional way based on enumeration or iterative methods with linear rate of convergence. Once the optimal control is determined, the cost-to-go function and its gradient with respect to the state variables is calculated to be used at the next stage. The proposed technique permits the efficient optimization of stochastic systems whose high dimensionality does not permit solution under the conventional DDP framework and for which successive approximation methods are not directly applicable due to stochasticity. Results for a four-reservoir example are presented. The purpose of this paper is to present a new computational algorithm for the stochastic optimization of sequential decision problems. One important and extensively studied class of such problems in the area of water resources is the discrete time optimal control of multireservoir systems under stochastic inflows. Other applications include the optimal design and operation of sewer systems [e.g., Mays and Wenzel, 1976; Labadie et al., 1980], the optimal conjunctive utilization of surface and groundwater resources [e.g., Buras, 1972], and the minimum cost water quality maintenance in rivers [e.g., Dracup and Fogarty, 1974; Chang and Yeh, 1973], to mention only a few of the water resources applications and pertinent references. An extensive review of dynamic programming applications in water resources can be found in the works by Yakowitz [1982] and Yeh [1985]. Before we proceed with the

Extensions to Linear Scheduling Optimization

Abstract: A two-state-variable, N-stage dynamic programming formulation of the liner scheduling problem is presented. The state variables are vectors. For any one activity, the first state variable represents a set of possible durations required to complete work at each of the locations. Likewise, for any one activity, the second state variable represents a set of possible interrupt durations between work performed at adjacent locations. Choices of activity duration and interrupt duration vectors are considered for each of the activities involved in a project. The problem is formulated within a conventional dynamic programming framework with the objective of minimizing the overall project duration. The methodology accounts for several of the realities of repetitive construction, including generialized precedence relationships and the ability to treat a variety of work continuity constraints. In addition, a sensitivity analysis procedure is described which permits the identification of near-optimal solutions, providing the user with schedule alternatives that might suit additional non-quantifiable criteria better. The Selinger bridge construction example is used to illustrate appication of the two-state-variable formulation and sensitivity analysis procedure.

Application of heuristic search and information theory to sequential fault diagnosis

Abstract: The problem of constructing optimal and near-optimal test sequences to diagnose permanent faults in electronic and electromechanical systems is considered. The test sequencing problem is formulated as an optimal binary AND/OR decision tree construction problem, whose solution is known to be NP-complete. The approach is based on integrating concepts from information theory and heuristic AND/OR graph search methods to subdue the computational explosion of the optimal test sequencing problem. Lower bounds on the optimal cost-to-go are derived from the information-theoretic concepts of Huffman coding and entropy, which ensure that an optimal solution is found using the heuristic AND/OR graph search algorithms. This makes it possible to obtain optimal test sequences to problems that are intractable with the traditional dynamic programming techniques. In addition, a class of test sequencing algorithms that provide a tradeoff between optimality and complexity have been derived using the epsilon -optimal and limited search strategies. The effectiveness of the algorithms is demonstrated on several test cases. As a by-product, this approach to test sequencing can be adapted to solve a wide variety of binary identification problems arising in other fields. >

An OT2 Algorithm for the NI/G/NI/ND Capacitated Lot Size Problem

Abstract: In this paper, we study a class of the capacitated dynamic lot size problem, where, over time, the setup costs are nonincreasing, the unit holding costs have arbitrary pattern, the unit production costs are nonincreasing and the capacities are nondecreasing. We investigate the properties of the optimal solution for the problem and develop the concept of candidate subplan. It is proven that only the candidate subplans need to be examined in searching for an optimal solution. A dynamic programming algorithm, incorporating the concept of candidate subplan, is then devised which has run time complexity of OT2.

A variational dynamic programming approach to robot-path planning with a distance-safety criterion

Abstract: An approach to robot-path planning is developed by considering both the traveling distance and the safety of the robot. A computationally-efficient algorithm is developed to find a near-optimal path with a weighted distance-safety criterion by using a variational calculus and dynamic programming (VCDP) method. The algorithm is readily applicable to any factory environment by representing the free workspace as channels. A method for deriving these channels is also proposed. Although it is developed mainly for two-dimensional problems, this method can be easily extended to a class of three-dimensional problems. Numerical examples are presented to demonstrate the utility and power of this method. >

On stochastic programming. II: Dynamic problems under risk

Abstract: This paper surveys recent work on dynamic stochastic programming problems and their applications. New results are included on the measurability and interpretation-in terms of the expected value of perfect information (EVPI)-of the dual multiplier processes corresponding to these problems. A final section reports preliminary computational experiments with algorithms for 2-stage problems

Speeding up dynamic programming

Abstract: A number of important computational problems in molecular biology, geology, speech recognition, and other areas can be expressed as recurrences which have typically been solved with dynamic programming. By using more sophisticated data structures, and by taking advantage of further structure from the applications, the authors speed up the computation of several of these recurrences by one or two orders of magnitude. The algorithms used are simple and practical. >

Application of dynamic programming and other mathematical techniques to pavement management systems

Abstract: This paper describes the application of improved mathematical techniques to the PAVER and Micro PAVER Pavement Management Systems. The use of stochastic dynamic programming to determine optimal strategies and related mean costs over specified life-cycle periods is outlined. The incorporation of simple simulation techniques to estimate the variance associated with these costs is described. The suitability of this approach for investigating the effects of deferred maintenance is presented. The use of outputs from these programs in subsequent prioritization and budget allocation modules is briefly discussed. An example that incorporates outputs from the dynamic programming and simulation programs is shown, and the validity of these outputs is discussed.

Differential dynamic programming and Newton's method

Abstract: A modified version of the original differential dynamic programming (DDP) algorithm for unconstrained discrete optimal control problems is described. This version, which differs from the original b...

Minimum time trajectory planner for the discrete dynamic robot model with dynamic constraints

Abstract: A minimum time trajectory planner is proposed for a manipulator arm. A totally discrete approach is adopted, in contrast to other models which use continuous-time but resort to discretization in the computation. The Neuman and Tourassis discrete-dynamic robot model is used to model the robot dynamics. The proposed trajectory planner includes joint-torque constraints to fully utilize the joint actuators. Realistic constraints such as the joint-jerk and joint-velocity constraints are incorporated into the model. The nonlinear optimization problem associated with the planner is partially linearized, which enables the iterative method of approximate programming to be used in solving the problem. Numerical examples for a two-link revolute arm are presented to demonstrate the use of the proposed trajectory planner. It is numerically verified that the convergence of the iterative algorithm is quadratic, and the trajectory planner therefore is computationally efficient. The use of a near-minimum time-cost function is also shown to yield a solution close to that obtained with the true minimum time-cost function. >

Smoothing noisy data using dynamic programming and generalized cross-validation.

Abstract: Smoothing and differentiation of noisy data using spline functions requires the selection of an unknown smoothing parameter. The method of generalized cross-validation provides an excellent estimate of the smoothing parameter from the data itself even when the amount of noise associated with the data is unknown. In the present model only a single smoothing parameter must be obtained, but in a more general context the number may be larger. In an earlier work, smoothing of the data was accomplished by solving a minimization problem using the technique of dynamic programming. This paper shows how the computations required by generalized cross-validation can be performed as a simple extension of the dynamic programming formulas. The results of numerical experiments are also included.

A dynamic programming solution to the dynamic, multi-item, single-machine scheduling

Abstract: This article presents a dynamic programming algorithm for scheduling, on a single machine, production of multiple items with time-varying deterministic demands. We formulate the scheduling problem with the objective of minimizing the sum of changeover and inventory holding costs. The formulation is appealing in that it represents changeover costs directly instead of by the familiar approximate technique of including setup costs in the objective. Our algorithm, which we developed using an approach similar to C. R. Glassey's that minimizes the total number of changeovers, casts the optimal schedule as a shortest path through a network embedded in a state space. It generates optimal schedules under two assumptions. First, we assume that in each time period within the planning horizon, the machine must either be shut down or be producing some one item for the entire time period. Second, we assume that inventory holding costs are representable as a nondecreasing function of aggregate inventory. We provide a nu...

Application of principal component analysis to long‐term reservoir management

Abstract: Determining the optimal long-term operating policy of a multireservoir power system requires solution of a stochastic nonlinear programing problem. For small systems the solution can be found by dynamic programing, but for large systems, no direct solution method exists yet, so that one must resort to mathematical manipulations to solve the problem. This paper presents a very efficient procedure for the case where high correlation exists between the reservoirs' trajectories and hence between the state variables. The procedure consists of performing principal component analysis (PCA) on the trajectories to find a reduced model of the system. The reduced model is then substituted into the operating problem, and the resulting problem is solved by stochastic dynamic programing. The reservoir trajectories on which the PCA is performed can be obtained by solving the operating problem deterministically for a large number of equally likely flow sequences. The results of applying the manipulation to Quebec's La Grande river, which has five reservoirs, are reported.

Scheduling, Routing, and Flow Control in Stochastic Networks

Abstract: Queueing models are frequently helpful in the analysis and control of communication, manufacturing, and transportation systems The theory of Markov decision processes and the inductive techniques of dynamic programming have been used to develop normative models for optimal control of admission, servicing, routing, and scheduling of jobs in queues and networks of queues We review some of these models, beginning with single-facility models and then progressing to models for networks of queues We emphasize the use of induction on a sequence of successive approximations of the optimal value function (value iteration) to establish the form of optimal control policies

Depth first search and dynamic programming algorithms for efficient CMOS cell generation

Abstract: An algorithmic framework is presented for mapping CMOS circuit diagrams into area-efficient, high-performance layouts in the style of one-dimensional transistor arrays. Using efficient search techniques and accurate evaluation methods, the huge solution space that is typical to such problems is transversed extremely fast, yielding designs of hand-layout quality. In addition to generating circuits that meet prespecified layout constraints in the context of a fixed target image, on-the-fly optimizations are performed to meet secondary optimization criteria. A practical dynamic programming routing algorithm is utilized to accommodate the special conditions that arise in this context. This algorithm has been implemented and is currently used at IBM for cell-library generation. >

Route selection through a dynamic ice field using the maximum principle

Abstract: To aid in the extension of the short navigating season in the Canadian Artic a route selection optimization procedure has been developed. This route selection procedure operates on the strategic navigation support level which has a spatial scale of several hundred kilometres and a decision horizon of 3 to 4 days. The optimization procedure is formulated as an optimal control problem and is solved by a synthesis of the maximum principle and the dynamic programming method. The potential of this optimization procedure is demonstrated by an application to two hypothetical route selection problems and one field example.