Showing papers on "Dynamic programming published in 1989"

Recursive methods in economic dynamics

Abstract: I. THE RECURSIVE APPROACH 1. Introduction 2. An Overview 2.1 A Deterministic Model of Optimal Growth 2.2 A Stochastic Model of Optimal Growth 2.3 Competitive Equilibrium Growth 2.4 Conclusions and Plans II. DETERMINISTIC MODELS 3. Mathematical Preliminaries 3.1 Metric Spaces and Normed Vector Spaces 3.2 The Contraction Mapping Theorem 3.3 The Theorem of the Maximum 4. Dynamic Programming under Certainty 4.1 The Principle of Optimality 4.2 Bounded Returns 4.3 Constant Returns to Scale 4.4 Unbounded Returns 4.5 Euler Equations 5. Applications of Dynamic Programming under Certainty 5.1 The One-Sector Model of Optimal Growth 5.2 A "Cake-Eating" Problem 5.3 Optimal Growth with Linear Utility 5.4 Growth with Technical Progress 5.5 A Tree-Cutting Problem 5.6 Learning by Doing 5.7 Human Capital Accumulation 5.8 Growth with Human Capital 5.9 Investment with Convex Costs 5.10 Investment with Constant Returns 5.11 Recursive Preferences 5.12 Theory of the Consumer with Recursive Preferences 5.13 A Pareto Problem with Recursive Preferences 5.14 An (s, S) Inventory Problem 5.15 The Inventory Problem in Continuous Time 5.16 A Seller with Unknown Demand 5.17 A Consumption-Savings Problem 6. Deterministic Dynamics 6.1 One-Dimensional Examples 6.2 Global Stability: Liapounov Functions 6.3 Linear Systems and Linear Approximations 6.4 Euler Equations 6.5 Applications III. STOCHASTIC MODELS 7. Measure Theory and Integration 7.1 Measurable Spaces 7.2 Measures 7.3 Measurable Functions 7.4 Integration 7.5 Product Spaces 7.6 The Monotone Class Lemma

Parallel and distributed computation

Abstract: This book focuses on numerical algorithms suited for parallelization for solving systems of equations and optimization problems Emphasis on relaxation methods of the Jacobi and Gauss-Seidel type, and issues of communication and synchronization Topics covered include: Algorithms for systems of linear equations and matrix inversion; Herative methods for nonlinear problems; and Shortest paths and dynamic programming

A tool for multiple sequence alignment

Abstract: Multiple sequence alignment can be a useful technique for studying molecular evolution and analyzing sequence-structure relationships. Until recently, it has been impractical to apply dynamic programming, the most widely accepted method for producing pairwise alignments, to comparisons of more than three sequences. We describe the design and application of a tool for multiple alignment of amino acid sequences that implements a new algorithm that greatly reduces the computational demands of dynamic programming. This tool is able to align in reasonable time as many as eight sequences the length of an average protein.

Bayesian Approach to Global Optimization: Theory and Applications

Abstract: 1 Global optimization and the Bayesian approach.- 1.1 What is global optimization?.- 1.2 Advantages of the Bayesian approach to global optimization.- 2 The conditions of Bayesian optimality.- 2.1 Introduction.- 2.2 Reduction to dynamic programming equations.- 2.3 The existence of a measurable solution.- 2.4 The calculation of conditional expectations.- 2.5 The one-step approximation.- 2.6 The adaptive Bayesian approach.- 3 The axiomatic non-probabilistic justification of Bayesian optimality conditions.- 3.1 Introduction.- 3.2 The linearity of the loss function.- 3.3 The existence of the unique a priori probability corresponding to subjective preferences.- 3.4 Optimal method under uncertainty.- 3.5 Nonlinear loss functions.- 4 Stochastic models.- 4.1 Introduction.- 4.2 Sufficient convergence conditions.- 4.3 The Gaussian field.- 4.4 Homogeneous Wiener field.- 4.5 A case of noisy observations.- 4.6 Estimation of parameters from dependent observations.- 5 Bayesian methods for global optimization in the Gaussian case.- 5.1 The one-step approximation.- 5.2 Adaptive models.- 5.3 Extrapolation models.- 5.4 Maximum likelihood models.- 5.5 The comparison of algorithms.- 5.6 The Bayesian approach to global optimization with linear constraints.- 5.7 The Bayesian approach to global optimization with nonlinear constraints.- 5.8 The Bayesian approach to multi-objective optimization.- 5.9 Interactive procedures and the Bayesian approach to global optimization.- 5.10 The reduction of multi-dimensional data.- 5.11 The stopping rules.- 6 The analysis of structure and the simplification of the optimization problems.- 6.1 Introduction.- 6.2 Structural characteristics and the optimization problem.- 6.3 The estimation of structural characteristics.- 6.4 The estimation of a simplification error.- 6.5 Examples of the estimates.- 7 The Bayesian approach to local optimization.- 7.1 Introduction.- 7.2 The one-dimensional Bayesian model.- 7.3 Convergence of the local Bayesian algorithm.- 7.4 Generalization of a multi-dimensional case.- 7.5 Convergence in the multi-dimensional case.- 7.6 The local Bayesian algorithm.- 7.7 Results of computer simulation.- 8 The application of Bayesian methods.- 8.1 Introduction.- 8.2 The optimization of an electricity meter.- 8.3 The optimization of vibromotors.- 8.4 The optimization of a shock-absorber.- 8.5 The optimization of a magnetic beam deflection system.- 8.6 The optimization of small aperture coupling between a rectangular waveguide and a microstrip line.- 8.7 The maximization of LSI yield by optimization of parameters of differential amplifier functional blocks.- 8.8 Optimization of technology to avoid waste in the wet-etching of printed circuit boards in iron-copper-chloride solutions.- 8.9 The optimization of pigment compounds.- 8.10 The least square estimation of electrochemical adsorption using observations of the magnitude of electrode impedance.- 8.11 Estimation of parameters of the immunological model.- 8.12 The optimization of nonstationary queuing systems.- 8.13 The analysis of structure of the Steiner problem.- 8.14 The estimation of decision making by intuition on the example of the Steiner problem.- 9 Portable FORTRAN software for global optimization.- 9.1 Introduction.- 9.2 Parameters.- 9.3 Methods available.- 9.4 Common blocks.- 9.5 The function.- 9.6 The main program.- 9.7 The example of the main program.- 9.8 Description of routines.- 9.9 BAYES1, the global Bayesian method by Mockus.- 9.10 UNT, the global method of extrapolation type by Zilinskas.- 9.11 LPMIN, the global method of uniform search by Sobolj, Shaltenis and Dzemyda.- 9.12 GLOPT, the global method of clustering type by Torn.- 9.13 MIG1, the global method of Monte Carlo (uniform random search).- 9.14 MIG2, the modified version of MIG 1.- 9.15 EXTR, the global one-dimensional method by Zilinskas.- 9.16 MIVAR4, the local method of variable metrics by Tieshis.- 9.17 REQP, the local method of recursive quadratic programming by Biggs.- 9.18 FLEXI, the local simplex method by Nelder and Mead.- 9.19 LBAYES, the local Bayesian method by Mockus.- 9.20 ANAL1, the method of analysis by structure by Shaltenis.- 9.21 Portability routines.- References.- Appendix 1 The software for global optimization for IMB/PC/XT/AT and compatibles.- Appendix 2 How the global optimization software can improve the performance of your CAD system.- Appendix 3 Machine dependent constants of portable FORTRAN.

Dynamic network traffic assignment considered as a continuous time optimal control problem

Abstract: Two continuous time formulations of the dynamic traffic assignment problem are considered, one that corresponds to system optimization and the other to a version of user optimization on a single mode network using optimal control theory. Pontryagin's necessary conditions are analyzed and given economic interpretations that correspond to intuitive notions regarding dynamic system optimized and dynamic user optimized traffic flow patterns. Notably, we offer the first dynamic generalization of Beckmann's equivalent optimization problem for static user optimized traffic assignment in the form of an optimal control problem. The analysis further establishes that a constraint qualification and convexity requirements for the Hamiltonian, which together ensure that the necessary conditions are also sufficient, are satisfied under commonly encountered regularity conditions.

Allocating modules to processors in a distributed system

Abstract: The author studies the complexity of the problem of allocating modules to processes in a distributed system to minimize total communication and execution costs. He shows that unless P=NP, there can be no polynomial-time epsilon -approximate algorithm for the problem, nor can there exist a local search algorithm that requires polynomial time per iteration and yields an optimum assignment. Both results hold even if the communication graph is planar and bipartite. On the positive side, it is shown that if the communication graph is a partial k-tree or an almost-tree with parameter k, the module allocation problem can be solved in polynomial time. >

Implementation of a Lagrangian relaxation based unit commitment problem

Abstract: The unit commitment problem in a power system involves determining a start-up and shut-down schedule of units to be used to meet the forecasted demand, over a future short term (24-168 hour) period. In solving the unit commitment problem, generally two basic decisions are involved. The "unit commitment" decision involves determining which generating units are to be running during each hour of the planning horizon, considering system capacity requirements including reserve, and the constraints on the start up and shut down of units. The related "economic dispatch" decision involves the allocation of system demand and spinning reserve capacity among the operating units during each specific hour of operation. As these two decisions are interrelated, the unit commitment problem generally embraces both these decisions, and the objective is to obtain an overall least cost solution for operating the power system over the scheduling horizon. The unit commitment problem belongs to the class of complex combinatorial optimization problems. During the past decade a new approach named "Lagrangian Relaxation" has been evolving for generating efficient solutions for this class of problems. It derives its name from the well-known mathematical technique of using Lagrange multipliers for solving constrained optimization problems, but is really a decomposition technique for the solution of large scale mathematical programming problems. The Lagrangian relaxation methodology generates easy subproblems for deciding commitment and generation schedules for single units over the planning horizon, independent of the commitment of other units.

The stochastic knapsack problem

Abstract: The problem of packing a knapsack of integer volume F with objects from K different classes to maximize profit is studied. Optimization is carried out over the class of coordinate convex policies. For the case of K=2, it is shown for a wide range of parameters that the optimal control is of the threshold type. In the case of Poisson arrivals and of knapsack and object volumes being integer multiples of each other, it is shown that the optimal policy is always of the double-threshold type. An O(F) algorithm to determine the revenue of threshold policies is also given. For the general case of K classes, the problem of the optimal static control where for each class a portion of the knapsack is dedicated is considered. An efficient finite-stage dynamic programming algorithm for locating the optimal static control is presented. Furthermore, variants of the optimal static control which allow some sharing among classes are also discussed. >

Methods of dynamic and nonsmooth optimization

Abstract: Nonsmooth Analysis and Geometry The Basic Problem in the Calculus of Variations Verification Functions and Dynamic Programming Optimal Control References.

Adaptive aggregation methods for infinite horizon dynamic programming

Abstract: A class of iterative aggregation algorithms for solving infinite horizon dynamic programming problems is proposed. The idea is to interject aggregation iterations in the course of the usual successive approximation method. An important feature that sets this method apart from earlier ones is that the aggregate groups of states change adaptively from one aggregation iteration to the next, depending on the progress of the computation. This allows acceleration of convergence in difficult problems involving multiple-ergodic classes for which methods using fixed groups of aggregate states are ineffective. No knowledge of special problem structure is utilized by the algorithms. >

A Dynamic Programming Model of Retirement Behavior

Abstract: This paper formulates a model of retirement behavior based on the solution to a stochastic dynamic programming problem. The workers objective is to maximize expected discounted utility over his remaining lifetime. At each time period the worker chooses how much to consume and whether to work full-time, part-time, or exit the labor force. The model accounts for the sequential nature f the retirement decision problem, and the role of expectations of uncertain future variables such as the worker's future lifespan, health status, marital and family status, employment status, as well as earnings from employment, assets, and social security retirement, disability and medicare payments. This paper applies a "nested fixed point" algorithm that converts the dynamic programming problem into the problem of repeatedly recomputing the fixed point to a contraction mapping operator as a subroutine of a standard nonlinear maximum likelihood program. The goal of the paper is to demonstrate that a fairly complex and realistic formulation of the retirement problem can be estimated using this algorithm and a current generation supercomputer, the Cray-2.

Trees, stars, and multiple biological sequence alignment

Abstract: One important problem in biological sequence comparison is how to simultaneously align several nucleic acid or protein sequences. A multiple alignment avoids possible inconsistencies among several pairwise alignments and can elucidate relationships not evident from pairwise comparisons. The basic dynamic programming algorithm for optimal multiple sequence alignment requires too much time to be practical for more than three sequences, the length of an average protein. Recently, Carrillo and Lipman (SIAMJ. Appl. Math., 48 (1988), pp. 1073–1082) have rendered feasible the optimal simultaneous alignment of as many as six sequences by showing that a consideration of minimal pairwise alignment costs can vastly decrease the number of cells a dynamic programming algorithm need consider. Their argument, however, requires the cost of a multiple alignment to be a weighted sum of the costs of its projected pairwise alignments.This paper presents an extension of Carrillo and Lipman's algorithm to the definition of mul...

Efficient dynamic programming implementations of Newton's method for unconstrained optimal control problems

Abstract: Naive implementations of Newton's method for unconstrainedN-stage discrete-time optimal control problems with Bolza objective functions tend to increase in cost likeN 3 asN increases. However, if the inherent recursive structure of the Bolza problem is properly exploited, the cost of computing a Newton step will increase only linearly withN. The efficient Newton implementation scheme proposed here is similar to Mayne's DDP (differential dynamic programming) method but produces the Newton step exactly, even when the dynamical equations are nonlinear. The proposed scheme is also related to a Riccati treatment of the linear, two-point boundary-value problems that characterize optimal solutions. For discrete-time problems, the dynamic programming approach and the Riccati substitution differ in an interesting way; however, these differences essentially vanish in the continuous-time limit.

Assembly line balancing with parallel workstations and dead time

Abstract: SUMMARY This paper presents a dynamic programming (DP) algorithm for solving an assembly line-balancing )ALB( problem with parallel workstations. Solutions represent a trade-off between the minimum number of stations required to achieve a balance and the cost of installing additional facilities. Both task costs and equipment costs are considered. A second feature of the algorithm is that it takes into account unproductive time during a cycle. The advantage of the DP approach is that it readily permits the recursive relationship associated with the serial ALB problem to be modified to accommodate the cost of paralleling. Solutions are obtained with an enumeration scheme that exploits known lower bounds. Two examples, and the results from a series of industry-based test problems, are presented to highlight the computations. In general, the algorithm performs well, but runs into trouble when the order strength of the underlying precedence graph is close to one or zero.

Maximum likelihood sequence estimators: a geometric view

Abstract: Communication issues are described in terms of macro operations between the data and the observation spaces. The problem of recovering the data is related to the inversion of an operator (the channel mapping); for this reason results available in linear algebra and functional analysis are applicable. Traditional concepts in communications are identified with these operations. An approach to the maximum-likelihood sequence estimator based on a sufficient statistic derived from these concepts is proposed. Intersymbol interference is removed by linear equalization, and a Viterbi-like dynamic programming algorithm takes into account the correlated noise in the metric evaluation. The performance of suboptimal receivers obtained by means of metric simplification is analyzed. >

Incorporating a multi-criteria decision procedure into the combined dynamic programming/production simulation algorithm for generation expansion planning

Abstract: A multi-objective optimization approach to utility generation expansion planning (GEP) is proposed. The approach is designed by adding a new multi-criteria decision (MCD) procedure to the conventional algorithm [1, 2] of Dynamic Programming (DP) combined with production simulation (DP/ production-simulation) using Cumulant method [3]. Section I of this paper gives a brief review of the presently existing multi-objective optimization approaches to GEP, showing that all these approaches use rough reserve margin and over-simplified merit-order techniques to calculate system reliability and estimate power production. In contrast, our approach using Cumulant method can simulate system production probabilistically. Section II presents our model formulation and derives the DP forward recursive formula for this model. Three MCD problems embedded within the model are identified: one decides the production order of system generating units for each DP state before production simulation; the other two select the near-best compromise state transition path for the GEP problem. Section III presents the proposed MCD procedure which combining a specially designed questionnaire, the Eigenvector method [4], and the Technique for Order Preference by Similarity to Ideal Solution [5], can help DM's distinguish the relative importance of multiple attributes associated with decisions of these three MCD problems, and thereby find their near-best compromise solutions. The new approach has been practically applied to a GEP study conducted within a project for feasibility evaluation of the fourth nuclear power plant of Taiwan, comprising the system's seventh and eighth nuclear units.

Trajectory Optimization with Risk Minimization for Military Aircraft

Abstract: Methods of time-controlled optimal flight-trajectory generation are developed that include the effects of risk from a threat environment. Lateral and vertical algorithms are developed for military jet aircraft with the intent of near real-time application. Simple analytic functions are used for threat models, as the focus of this paper is on general problem formulation and not detailed solutions for specific threats. A constant altitude, lateral flight-trajectory generation method is developed that optimizes with respect to time, fuel, final position, and risk exposure. Existing vertical plane trajectory generation methods that use standard direct operating costs of time and fuel are modified to include the effect of risks. Singular perturbation methods are used to obtain reduced-order airplane models that allow static rather than dynamic optimization. Pontryagin's Minimum Principle is used with a Fibonacci search method to minimize the cost functional. Formulation and numerical results are presented for both the horizontal and vertical plane problems. N the 1970's, rising fuel prices and improved micropro- cessor capability made consideration of onboard flight- path optimization appealing as a way to reduce operating costs. More recently, increased air traffic and the associated control problems have shifted research emphasis toward four- dimensional trajectory optimization. The four-dimensional problem typically includes a cost of time that allows trajecto- ries to meet required times of destination arrival. The majority of research in this area has been directed toward commercial operations and involves using the energy state approximation for generating vertical flight paths. It has been shown1^ that singular perturbation theory (SPT) may be used successfully to reduce the order of this problem. A reduced-order, often static, optimization technique can then be employed that greatly reduces computational burden. In this paper, methods of trajectory generation are developed that expand the basic SPT techniques to address the needs of military operations. Trajectory generation will be a fundamental requirement for future military aircraft flight management systems. These systems will be required to take advantage of all available information to perform integrated task processing and reduce pilot workload. In addition, the systems should be able to provide updates at any time throughout a mission or at regular time intervals sufficient for threat avoidance. Much of the previous work in trajectory generation for military air- craft5"7 has concentrated on feasible directions algorithms that use dynamic programming. These methods tend to be computationally intense and, therefore, are not well suited for onboard applications in dynamic threat environments. An ideal flight trajectory for military operations meets the mission requirements within the constraints of the aircraft limitations while minimizing exposure to threats. Several lev- els of information are used in the selection of such a strategic flight path and velocity profile. The trajectory will be a function of mission requirements (time of arrival, point of arrival, etc.), aircraft performance limitations (for example, fuel quantity and thrust limits), and the threat environment. The threat models used in this study do not refer to any specific military scenario. They are simple analytic functions

A solution to the multiple set-up problem with dynamic demand

Abstract: In this note, we study an important practical problem, the classic dynamic lot size model with set-up cost including a fixed cost and freight cost, where the freight cost is proportional to the number of containers used. This freight cost is the additional multiple set-up cost. We solve the T-period problem in O(T44) effort using dynamic programming, Handled by the Editor-in-Chief.

Integrating expert systems with dynamic programming in generation expansion planning

Abstract: Interactive software developed for integrating engineering experience and judgment from the planning department with a powerful mathematical optimization method is described. The excessive size of the state space generated by conventional multidimensional dynamic programming is reduced to manageable proportions by rule-based procedures for implementing windows in state space and controls in policy state. Project frames describing generation options and state frames describing future conditions of the system are established and manipulated by rules. Dynamic programming simultaneously tracks a feasible set of suboptimal scenarios. The program is interactive and is written in Prolog with numerically intensive portions in C. >

A dynamic programming approach to stochastic assembly line balancing

Abstract: Consider the problem of minimizing the required number of work stations on an assembly line for a given cycle time when the processing times are independent, normally distributed random variables. The assignment of tasks to stations is subject to precedence conditions, caused by technological constraints, and a lower bound on the probability of the work at any station being completed within the cycle time. We present two dynamic programming (DP) algorithms for this problem, each guaranteed to be optimal under a certain mild condition. Our general approach is based on the Held et al. (Held, M., R. M. Karp, R. Shareshian. 1963. Assembly-line-balancing-dynamic programming with precedence constraints. Oper. Res. 11 442–459.) formulation of the deterministic line balancing problem and thus represents a modification of previous work by Kao (Kao, E. P. C. 1976. A preference order dynamic program for stochastic assembly line balancing. Management Sci. 22 1097–1104.). Computational results indicate that both algor...

Discrete Dynamic Programming and Viscosity Solutions of the Bellman Equation

Abstract: This paper presents a technique for approximating the viscosity solution of the Bellman equation in deterministic control problems. This technique, based on discrete dynamic programming, leads to monotonically converging schemes and allows to prove a priori error estimates. Several computational algorithms leading to monotone convergence are reviewed and compared.

Generalized Dynamic Programming for Stochastic Combinatorial Optimization

Abstract: In stochastic versions of combinatorial optimization problems, the objective is to maximize or minimize a function of random variables. For many problems of this type, conventionally applied dynamic programming DP may fail to generate an optimal solution due to the potential violation of the monotonicity assumption of DP. We develop a generalization of DP that guarantees optimality even in the absence of monotonicity. We illustrate the methodology on a version of the stochastic traveling salesman problem for which a previously proposed DP algorithm E. Kao is potentially suboptimal due to the violation of monotonicity M. Sniedovich. Using Generalized DP, we are able to modify the algorithm to guarantee optimality.

Control of Markov chains with long-run average cost criterion: the dynamic programming equations

Abstract: The long-run average cost control problem for discrete time Markov chains on a countable state space is studied in a very general framework. Necessary and sufficient conditions for optimality in terms of the dynamic programming equations are given when an optimal stable stationary strategy is known to exist (e.g., for the situations studied in [Stochastic Differential Systems, Stochastic Control Theory and Applications, IMA Vol. Math. App. 10, Springer-Verlag, New York, Berlin, 1988, pp. 57–77]). A characterization of the desired solution of the dynamic programming equations is given in a special case. Also included is a novel convex analytic argument for deducing the existence of an optimal stable stationary.strategy when that of a randomized one is known.

A heuristic approach to the three-dimensional cargo-loading problem

Abstract: SUMMARY Since no exact analytical method for solving the three-dimensional cargo-loading problem has been developed, the heuristic approaches with practical assumptions are still useful. A dynamic programming approach to this problem is proposed in this paper. Loading a three-dimensional cargo space is done layer by layer, a special property which is taken advantage of in the proposed algorithm. The computational performance of this heuristic is demonstrated by comparing its results with suggested values published by the General Services Administration, Washington, DC.