Showing papers on "Dynamic programming published in 1982"

Dynamic programming applications in water resources

Abstract: The central intention of this survey is to review dynamic programming models for water resource problems and to examine computational techniques which have been used to obtain solutions to these problems. Problem areas surveyed here include aqueduct design, irrigation system control, project development, water quality maintenance, and reservoir operations analysis. Computational considerations impose severe limitation on the scale of dynamic programming problems which can be solved. Inventive numerical techniques for implementing dynamic programming have been applied to water resource problems. Discrete dynamic programming, differential dynamic programming, state incremental dynamic programming, and Howard's policy iteration method are among the techniques reviewed. Attempts have been made to delineate the successful applications, and speculative ideas are offered toward attacking problems which have not been solved satisfactorily.

An optimal control approach to dynamic routing in networks

Abstract: This paper explores the application of optimal control theory to the problem of dynamic routing in networks. The approach derives from a continuous state space model for dynamic routing and an associated linear optimal control problem with linear state and control variable inequality constraints. The conceptual form of an algorithm is presented for finding a feedback solution to the optimal control problem when the inputs are assumed to be constant in time. The algorithm employs a combination of necessary conditions, dynamic programming, and linear programming to construct a set of convex polyhedral cones which cover the admissible state space with optimal controls. An implementable form of the algorithm, along with a simple example, is presented for a special class of single destination networks.

Spinning Reserve Constrained Static and Dynamic Economic Dispatch

Abstract: One of the recurring problems facing energy control center dispatchers each day is how to operate the system during the periods of high load pickup, such that there is sufflcient generation to follow the load pickup, while still maintaining reasonable reserve and/or regulation margin This paper shows a technical solution to this problem which can be achieved with a very efficient use of computer resources The problem is expressed as a dynamic programming scheduling problem, and a feasible, but suboptimal solution is proposed, which eliminates the usual search space problem This method reduces the problem to a backward sequence of dispatch problems, with the generator limits being carefully adjusted between each time interval in the solution sequence The paper also discusses an efficient algorithm for the solution of a reserve constrained economic dispatch, which is the static optimization technique used at each interval

On Dynamic Programming Methods for Assembly Line Balancing

Abstract: Two dynamic programming approaches for treating sequencing problems—one proposed by Schrage and Baker and the other by Lawler—are discussed in the context of an assembly line balancing problem. A variant of the Schrage-Baker method is proposed to extend its range of applicability. The three approaches are compared using randomly generated test problems. We find that Lawler's “reaching”-based approach is superior to the other two “pulling”-based alternatives in both time and storage requirements. Based on the empirical results, we present time and space estimates for solving problems of different sizes and order strengths, and discuss the relative merits of the three procedures.

Control, Identification, and Input Optimization

Abstract: I. Introduction.- 1. Introduction.- 1.1. Optimal Control.- 1.2. System Identification.- 1.3. Optimal Inputs.- 1.4. Computational Preliminaries.- Exercises.- II. Optimal Control and Methods for Numerical Solutions.- 2. Optimal Control.- 2.1. Simplest Problem in the Calculus of Variations.- 2.1.1. Euler-Lagrange Equations.- 2.1.2. Dynamic Programming.- 2.1.3. Hamilton-Jacobi Equations.- 2.2. Several Unknown Functions.- 2.3. Isoperimetric Problems.- 2.4. Differential Equation Auxiliary Conditions.- 2.5. Pontryagin's Maximum Principle.- 2.6. Equilibrium of a Perfectly Flexible Inhomogeneous Suspended Cable.- 2.7. New Approaches to Optimal Control and Filtering.- 2.8. Summary of Commonly Used Equations.- Exercises.- 3. Numerical Solutions for Linear Two-Point Boundary-Value Problems..- 3.1. Numerical Solution Methods.- 3.1.1. Matrix Riccati Equation.- 3.1.2. Method of Complementary Functions.- 3.1.3. Invariant Imbedding.- 3.1.4. Analytical Solution.- 3.2. An Optimal Control Problem for a First-Order System.- 3.2.1. The Euler-Lagrange Equations.- 3.2.2. Pontryagin's Maximum Principle.- 3.2.3. Dynamic Programming.- 3.2.4. Kalaba's Initial-Value Method.- 3.2.5. Analytical Solution.- 3.2.6. Numerical Results.- 3.3. An Optimal Control Problem for a Second-Order System.- 3.3.1. Numerical Methods.- 3.3.2. Analytical Solution.- 3.3.3. Numerical Results and Discussion.- Exercises.- 4. Numerical Solutions for Nonlinear Two-Point Boundary-Value Problems.- 4.1. Numerical Solution Methods.- 4.1.1. Quasilinearization.- 4.1.2. Newton-Raphson Method.- 4.2. Examples of Problems Yielding Nonlinear Two-Point Boundary-Value Problems.- 4.2.1. A First-Order Nonlinear Optimal Control Problem.- 4.2.2. Optimization of Functionals Subject to Integral Constraints.- 4.2.3. Design of Linear Regulators with Energy Constraints.- 4.3. Examples Using Integral Equation and Imbedding Methods.- 4.3.1. Integral Equation Method for Buckling Loads.- 4.3.2. An Imbedding Method for Buckling Loads.- 4.3.3. An Imbedding Method for a Nonlinear Two-Point Boundary-Value Problem.- 4.3.4. Post-Buckling Beam Configurations via an Imbedding Method.- 4.3.5. A Sequential Method for Nonlinear Filtering.- Exercises.- III. System Identification.- 5. Gauss-Newton Method for System Identification.- 5.1. Least-Squares Estimation.- 5.1.1. Scalar Least-Squares Estimation.- 5.1.2. Linear Least-Squares Estimation.- 5.2. Maximum Likelihood Estimation.- 5.3. Cramer-Rao Lower Bound.- 5.4. Gauss-Newton Method.- 5.5. Examples of the Gauss-Newton Method.- 5.5.1. First-Order System with Single Unknown Parameter.- 5.5.2. First-Order System with Unknown Initial Condition and Single Unknown Parameter.- 5.5.3. Second-Order System with Two Unknown Parameters and Vector Measurement.- 5.5.4. Second-Order System with Two Unknown Parameters and Scalar Measurement.- Exercises.- 6. Quasilinearization Method for System Identification.- 6.1. System Identification via Quasilinearization.- 6.2. Examples of the Quasilinearization Method.- 6.2.1. First-Order System with Single Unknown Parameter.- 6.2.2. First-Order System with Unknown Initial Condition and Single Unknown Parameter.- 6.2.3. Second-Order System with Two Unknown Parameters and Vector Measurement.- 6.2.4. Second-Order System with Two Unknown Parameters and Scalar Measurement.- Exercises.- 7. Applications of System Identification.- 7.1. Blood Glucose Regulation Parameter Estimation.- 7.1.1. Introduction.- 7.1.2. Physiological Experiments.- 7.1.3. Computational Methods.- 7.1.4. Numerical Results.- 7.1.5. Discussion and Conclusions.- 7.2. Fitting of Nonlinear Models of Drug Metabolism to Experimental Data.- 7.2.1. Introduction.- 7.2.2. A Model Employing Michaelis and Menten Kinetics for Metabolism.- 7.2.3. An Estimation Problem.- 7.2.4. Quasilinearization.- 7.2.5. Numerical Results.- 7.2.6. Discussion.- Exercises.- IV. Optimal Inputs for System Identification.- 8. Optimal Inputs.- 8.1. Historical Background.- 8.2. Linear Optimal Inputs.- 8.2.1. Optimal Inputs and Sensitivities for Parameter Estimation.- 8.2.2. Sensitivity of Parameter Estimates to Observations.- 8.2.3. Optimal Inputs for a Second-Order Linear System.- 8.2.4. Optimal Inputs Using Mehra's Method.- 8.2.5. Comparison of Optimal Inputs for Homogeneous and Nonhomogeneous Boundary Conditions.- 8.3. Nonlinear Optimal Inputs.- 8.3.1. Optimal Input System Identification for Nonlinear Dynamic Systems.- 8.3.2. General Equations for Optimal Inputs for Nonlinear Process Parameter Estimation.- Exercises.- 9. Additional Topics for Optimal Inputs.- 9.1. An Improved Method for the Numerical Determination of Optimal Inputs.- 9.1.1. Introduction.- 9.1.2. A Nonlinear Example.- 9.1.3. Solution via Newton-Raphson Method.- 9.1.4. Numerical Results and Discussion.- 9.2. Multiparameter Optimal Inputs.- 9.2.1. Optimal Inputs for Vector Parameter Estimation.- 9.2.2. Example of Optimal Inputs for Two-Parameter Estimation.- 9.2.3. Example of Optimal Inputs for a Single-Input, Two-Output System.- 9.2.4. Example of Weighted Optimal Inputs.- 9.3. Observability, Controllability, and Identifiability.- 9.4. Optimal Inputs for Systems with Process Noise.- 9.5. Eigenvalue Problems.- 9.5.1. Convergence of the Gauss-Seidel Method.- 9.5.2. Determining the Eigenvalues of Saaty's Matrices for Fuzzy Sets.- 9.5.3. Comparison of Methods for Determining the Weights of Belonging to Fuzzy Sets.- 9.5.4. Variational Equations for the Eigenvalues and Eigenvectors of Nonsymmetric Matrices.- 9.5.5. Individual Tracking of an Eigenvalue and Eigenvector of a Parametrized Matrix.- 9.5.6. A New Differential Equation Method for Finding the Perron Root of a Positive Matrix.- Exercises.- 10. Applications of Optimal Inputs.- 10.1. Optimal Inputs for Blood Glucose Regulation Parameter Estimation.- 10.1.1. Formulation Using Bolie Parameters for Solution by Linear or Dynamic Programming.- 10.1.2. Formulation Using Bolie Parameters for Solution by Method of Complementary Functions or Riccati Equation Method.- 10.1.3. Improved Method Using Bolie and Bergman Parameters for Numerical Determination of the Optimal Inputs.- 10.2. Optimal Inputs for Aircraft Parameter Estimation.- Exercises.- V. Computer Programs.- 11. Computer Programs for the Solution of Boundary-Value and Identification Problems.- 11.1. Two-Point Boundary-Value Problems.- 11.2. System Identification Problems.- References.- Author Index.

Speed-Up in Dynamic Programming

Abstract: Dynamic programming is a general problem-solving method that has been used widely in many disciplines, including computer science. In this paper we present some recent results in the design of efficient dynamic programming algorithms. These results illustrate two approaches for achieving efficiency: the first by developing general techniques that are applicable to a broad class of problems, and the second by inventing clever algorithms that take advantage of individual situations.

On some Control Problems for Queues

Abstract: This thesis consists of three parts. In the first one, optimal policies are constructed for some singe-line queueing situations. The second part deals with finite-state Markovian decision processes, and in the third part the practical modelling of a more complex problem is discussed and exemplified.The central control object of part I is an M!M/1 queue with fixed arrival rate and controllable service rate. The objective is to minimize the expected long-run average of a cost rate, which isa sum of two functions, associated with the queue length (the holding cost) and the service rate (the service cost), respectively. For the case of a fin ite waiting-room, terminal costs are constructed, such that a solution to the associated dynamic programming (Bellman) equation exists, which is affine in the time parameter. The corresponding optimal control is independent of both time and the length of the control interval. It hasa form which is subsequently used in generali zing into the case of an infinite waiting room. For this case, the analysis res ults in an efficient algorithm, and in several structural results. Assuming essentially only that the holding cost is increasing, it is proved that a monotone optimal policy exists, i.e. that the optimal choice of service rate is an in creasing function of the present queue length. Three variations of the ce ntral problem are also treated in part I. These are the M/M/c problem (for which the above monotonicity result holds only under a stronger condition), the problem of a controllable ar rival rate (with fixed service rate), and the discounted cost problem.In part II, finite-state Markovian decision processes are discussed. A brief and heuristic introduction is given, regarding continuous-time Markov chains, cost structures on these, and the problem of constructing an optimal poli cy. The purpose is to point out the relations to the queueing control problem with finite waiting-room. Counterexamples demonstrate that the approach of part I is not universally applicable.In part 111, a simplified mode! is discussed for a situation where th e customers may reenter the queue after a stochastic delay. It is argued that under heavy-traffic conditions, the influx of reentering customers can be approximated with the output of a linear stochastic system with state-dependent Gaussian noise, whose dynamics depend on the delay distribution. This idea is exemplified with the res ults from a simulated experiment on a telephone station.

A stochastic control approach to group preventive replacement in a multicomponent system

Abstract: A group preventive replacement problem is formulated in continuous time for a multicomponent system having identical elements. The dynamic programming equation is obtained in the framework of the theory of optimal control of jump processes. For a discrete time version of the model, the numerical computation of optimal and suboptimal strategies of group preventive replacement are done. A monotonicity property of the Bellman functional (or cost-to-go function) is used to reduce the size of the computational problem. Some counterintuitive properties of the optimal strategy are apparent in the numerical results obtained.

Multicriteria dynamic programming with an application to the integer case

Abstract: Fundamental dynamic programming recursive equations are extended to the multicriteria framework. In particular, a more detailed procedure for a general recursive solution scheme for the multicriteria discrete mathematical programming problem is developed. Definitions of lower and upper bounds are offered for the multicriteria case and are incorporated into the recursive equations to aid problem solution by eliminating inefficient subpolicies. Computational results are reported for a set of 0–1 integer linear programming problems.

Optimal Long Range Transmission Planning with AC Load Flow

Abstract: This paper describes a new procedure for the optimal long range expansion planning of a transmission network. In addition to providing an optimal plan, a set of cost competitive expansion plans can be generated. System expansion is specified in terms of discrete transmission lines and two kV levels. Economic impact of factors such as cost escalation, economy of scale, right-of-way unavailability, external system interconnections, etc., are incorporated in the formulation. A dynamic optimization procedure based on a nonlinear branch and bound algorithm is employed. The resulting optimal plan specifies when, where, and what type of construction should occur. A unique feature of the method, which reduces considerably the computational burden, is an automatic generation of alternatives algorithm and associated procedures for elimination of infeasible and non-optimal alternatives early in the process. Computational speed can also be affected by the user option of employing either dc or ac load flow calculations.

An adaptive, ordered, graph search technique for dynamic time warping for isolated word recognition

Abstract: The technique of dynamic time warping (DTW) is relied on heavily in isolated word recognition systems. The advantage of using DTW is that reliable time alignment between reference and test patterns is obtained. The disadvantage of using DTW is the heavy computational burden required to find the optimal time alignment path. Several alternative procedures have been proposed for reducing the computation of DTW algorithms. However, these alternative methods generally suffer from a loss of optimality or precision in defining points along the alignment path. In this paper we propose another alternative procedure for implementing a DTW algorithm. The procedure is based on the well-known class of techniques for a directed search through a grid to find the "shortest" path. An adaptive version of a directed search procedure is defined and shown to be capable of obtaining the exact DTW solution with reduced computation of distances but with increased overhead. It is shown that for machines where the time for distance computation is significantly larger than the time for combinatorics and overhead, a potential gain in speed of up to 3 : 1 can be realized with the directed search algorithm. Formal comparison of the directed search algorithm with a standard DTW method, in an isolated word recognition test, showed essentially no loss in recognition accuracy when the parameters of the directed search were selected to realize the 3:1 reduction in distance computation.

Dynamic Programming is Optimal for Nonserial Optimization Problems

Abstract: We consider discrete optimization problems in which the only exploitable feature of the objective function is a limited form of decomposability. “Nonoverlapping comparison algorithms” are defined as a model of procedures which decompose the problem and apply Bellman’s principle of optimality. Nonserial dynamic programming (DP), a simple elimination procedure, is shown to be optimal among all nonoverlapping comparison algorithms, including nondeterministic algorithms. These results can give an exponential lower bound on the shortest admissible proof that a solution is optimal. Furthermore, if part of the search space is ruled out, a subset of the comparisons made by DP optimally searches the remainder. We suggest that the running time of DP is a useful measure of the “interaction complexity” of a problem, and that because of its simplicity DP is of practical as well as theoretical interest.

Solving a multiobjective traveling salesman problem by dynamic programming

Abstract: In this paper Klotzler's method of multiobjective dynamic programming is applied to the solution of a two-dimensional traveling salesman problem. In this way Bellman's and Held/Karp's dynamic programming approach to one-dimensional traveling salesman problems is extended to the multiobjective case.

Discrete and Dynamic Decision Analysis

Abstract: Introduction to methods for structuring and solving decision problems. Beginning with a discussion of probability and problems of measurement, the first half of the text covers decision analysis and the use of discrete structures. The latter half is concerned with solving sequential problems through the use of dynamic programming. Each chapter includes exercises in formulating and numerically solving decision problems.

Dynamic time warping for isolated word recognition based on ordered graph searching techniques

Abstract: The technique of dynamic time warping (DTW) is relied on heavily in isolated word recognition systems. The advantage of using DTW is that reliable time alignment between reference and test patterns is obtained. The disadvantage of using DTW is the heavy computational burden required to find the optimal time alignment path. Several alternative procedures have been proposed for reducing the computation of DTW algorithms. However these alternative methods generally suffer from a loss of optimality or precision in defining points along the alignment path. In this paper we propose another alternative procedure for implementing a DTW algorithm. The procedure is based on the well known class of techniques for a directed search through a grid to find the "shortest' path. An adaptive version of a directed search procedure is defined and shown to be capable of obtaining the exact DTW solution with reduced computation of distances but with increased overhead. It is shown that for machines where the time for distance computation is significantly larger than the time for combinatorics and overhead, a potential gain in speed of up to 3 to 1 can be realized with the directed search algorithm. Formal comparison of the directed search algorithm with a standard DTW method, in an isolated word recognition test, showed essentially no loss in recognition accuracy when the parameters of the directed search were selected to realize the 3 to 1 reduction in distance computation.

The design of optimal DFT algorithms using dynamic programming

Abstract: A broad class of efficient, discrete Fourier transform algorithms is developed by partitioning short DFT algorithms into factors. The factored short DFT's are combined into longer DFT's using a prime factor algorithm (PFA). By exploiting a property which allows some of the factors to commute, a large set of possible DFT algorithms is generated which contains both the prime factor algorithm and the Winograd Fourier Transform Algorithm (WFTA) as special cases. The problem of finding an algorithm from this class which is optimal with respect to the specific add, multiply, and data transfer characteristics of a partlcular implementation is posed, and a highly effective dynamic programming algorithm is presented as a solution.

Electric-generation-expansion analysis system. Volume 1: solution techniques, computing methods, and results. Final report. [EGEAS computer code]

Abstract: EGEAS is a modular state-of-the-art capacity expansion software package. It contains five capacity expansion analysis options ranging from preliminary analysis tools based on screening curves and linear programming to sophisticated non-linear analysis tools utilizing a Generalized Benders' Decomposition algorithm and a Dynamic Programming algorithm. A stand alone, detailed probabilistic production costing algorithm is also available for prespecified expansion plan production cost and reliability analysis. The multiple EGEAS analysis options are incorporated into a single control analysis program allowing users to match the analysis option selected to a particular problem's requirements and complexity. At the same time, the implementation of a flexible modular and extendable data base common to all five analysis options simplifies data input and maintenance, and rules out input-related inconsistencies among the analysis options. The development and implementation of a detailed but nevertheless efficient probabilistic production costing algorithm made possible the addition to EGEAS of a number of advanced features. These features include maintenance scheduling, economy interchange and reserve sharing, storage and limited energy unit modeling as well as Non-Dispatchable Generation and Load Management analysis. Sensitivity, uncertainty and trade-off analyses are also available in EGEAS. The structure and capabilities of EGEAS as well as the results of amore » testing and validation effort exercise are presented in this Volume, No. 1.« less

Operations Research in Progress

Abstract: 1: Mathematical Programming and Optimal Control Theory.- An Optimality Condition and its Application to Parametric Semi-Infinite Optimization.- The Choice of a Parameter in a Penalty Method.- Recent Results on ?-Conjugation and Nonconvex Optimization.- On Quantitative Stability of Point-to-Set-Mappings and the Rate of Convergence of Corresponding Algorithms.- On the Penalization Method in Convex Stochastic Programming.- A New Algorithm of Solving the Flow - Shop Problem.- On Dynamic Traffic Assignment.- On an Approximation Problem of Mechanical Structural Optimization.- Optimal Daily Scheduling of the Electricity Production in Hungary.- Power Distribution Planning and the Application of Linear Mixed-Integer Programming.- Optimal Flood Control by Reservoir Systems Using the Reduced Gradient Method.- Instant Optimization of Hydro Energy Storage Plants.- Dynamic Programming in Power System Extension Planning.- Some New Multicriteria Approaches.- Equilibrium Selection in a Wage Bargaining Situation with Incomplete Information.- Planning and Forecast Horizons in a Simple Wheat Trading Model.- Intertemporal Reversales of Environmental and Macroeconomic Policies.- Optimal Control of Concave Economic Models with two Control Instruments.- Optimal Control with Switching Dynamics.- Dynamic Systems with Several Decision-Makers.- Optimal Bimodal Harvest Policies in Age-Specific Bioeconomic Models.- Growth Rates, Optimal Harvesting and Related Topics in the Mass Rearing of Tsetse Flies.- The Release of Partly Fertile Males or Females in the Application of the Sterile-Insect Technique: Mathematical Analysis of the Hard-Release Strategy.- 2: Stochastic Models.- New Developments in Optimal Control of Queueing Systems.- Estimation and Control in a GI|M|1-System.- On Discriminating among Stochastic Models - A Survey.- Increasing the Work-Safety in Nuclear Power Plants through the Use of Preventive Maintenance Policies.- Recent Developments in Econometrics.- Slight Misspecifications of Linear Systems.- Local Sensitivity Analysis and Matrix Derivatives.- Analysis and Forecasting of Demand for Electricity Using Time Series Analysis.- Short Term Load Predication in Electric Power Systems.- Interactive Short-Term Load Forecasting.- Predicting the Demand for Electricity - An Application of Transfer Function Analysis.- Problems Associated with the Design of a Reliability Model in Electricity Industry.

Locally constrained dynamic programming in automatic speech recognition

Abstract: Recent years have seen the emergence of dynamic programming as one of the most important tools available for overcoming temporal variability problems in automatic speech recognition. Considerable research effort has been devoted to the investigation of different dynamic programming algorithms, yet most work has been concerned with global, rather than local, constraints upon temporal variation. Clearly the likelihood of timescale distortion is not constant for the entire duration of an utterance; variability must be, at least to some extent, data dependent. It is therefore desirable that information to this effect should be made available to the dynamic time warping process. This paper describes a technique for obtaining an estimate of local timescale variability based on a bi-directional dynamic programming algorithm and basic fuzzy set theory. Results are presented which indicate that this technique will lead to improved discrimination, especially if the differences between classes are mainly due to temporal structure.

Computation of production control policies by a dynamic programming technique

Abstract: The problem of production management for an automated manufacturing system is described. The system consists of machines that can perform a variety of tasks on a family of parts. The machines are unreliable, and the main difficulty the control system faces is to meet production requirements while machines fail and are repaired at random times. A multi-level hierarchical control algorithm is proposed which involves a stochastic optimal control problem at the first level. Optimal production policies are characterized and a computational scheme is described.

Application of dynamic programming to multi-stage batch machining

Abstract: This paper presents an algorithm based on dynamic programming for the optimization of a series of manufacturing operations during multi-stage batch machining. The machine variables can be treated as discrete when necessary. The lower and upper limits of the overall cycle time are computed first. The interval between these limits is then divided into a sufficiently large number of parts to define a number of feasible cycle times. We then compute the optimum values of the process variables for each operation and the value of the chosen objective function corresponding to each of the cycle times. The cycle time and process variables of each operation are optimum corresponding to which objective function is optimized. The algorithm has been applied to a test problem. The results have been found to be directly usable and realistic.