Showing papers on "Dynamic programming published in 1997"

Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations

Abstract: Preface.- Basic notations.- Outline of the main ideas on a model problem.- Continuous viscosity solutions of Hamilton-Jacobi equations.- Optimal control problems with continuous value functions: unrestricted state space.- Optimal control problems with continuous value functions: restricted state space.- Discontinuous viscosity solutions and applications.- Approximation and perturbation problems.- Asymptotic problems.- Differential Games.- Numerical solution of Dynamic Programming.- Nonlinear H-infinity control by Pierpaolo Soravia.- Bibliography.- Index

Algebra of programming

Abstract: 1. Programs. 2. Functions and Categories. 3. Applications. 4. Relationships and Allegories. 5. Datatypes in Allegories. 6. Optimisation Problems. 7. Thinning Algorithms. 8. Dynamic Programming. 9. Greedy Algorithms. Appendices.

Heuristic and randomized optimization for the join ordering problem

Abstract: Recent developments in database technology, such as deductive database systems, have given rise to the demand for new, cost-effective optimization techniques for join expressions. In this paper many different algorithms that compute approximate solutions for optimizing join orders are studied since traditional dynamic programming techniques are not appropriate for complex problems. Two possible solution spaces, the space of left-deep and bushy processing trees, are evaluated from a statistical point of view. The result is that the common limitation to left-deep processing trees is only advisable for certain join graph types. Basically, optimizers from three classes are analysed: heuristic, randomized and genetic algorithms. Each one is extensively scrutinized with respect to its working principle and its fitness for the desired application. It turns out that randomized and genetic algorithms are well suited for optimizing join expressions. They generate solutions of high quality within a reasonable running time. The benefits of heuristic optimizers, namely the short running time, are often outweighed by merely moderate optimization performance.

Energy minimization using multiple supply voltages

Abstract: We present a dynamic programming technique for solving the multiple supply voltage scheduling problem in both nonpipelined and functionally pipelined data-paths. The scheduling problem refers to the assignment of a supply voltage level (selected from a fixed and known number of voltage levels) to each operation in a data flow graph so as to minimize the average energy consumption for given computation time or throughput constraints or both. The energy model is accurate and accounts for the input pattern dependencies, re-convergent fanout induced dependencies, and the energy cost of level shifters. Experimental results show that using three supply voltage levels on a number of standard benchmarks, an average energy saving of 40.19% (with a computation time constraint of 1.5 times the critical path delay) can be obtained compared to using a single supply voltage level.

Controlled Optimization of Phases at an Intersection

Abstract: This paper presents a general purpose algorithm for real-time traffic control at an intersection. Our methodology, based on dynamic programming, allows optimization of a variety of performance indices such as delay, stops and queue lengths. Furthermore, optimal phase sequencing is a direct by-product of this new approach. These features make the new methodology a powerful tool for intersection control. We demonstrate the usefulness of the approach by a simulation experiment in which our intersection control algorithm is interfaced with a well established simulation package called TRAF-NETSIM. Our study compares the controlled optimization of phases methodology with fully-actuated as well as semi-actuated control. We show that consistent reductions in delay may be possible by adopting the new algorithm.

A Minimal Algorithm for the 0-1 Knapsack Problem

Abstract: Several types of large-sized 0-1 Knapsack Problems (KP) may be easily solved, but in such cases most of the computational effort is used for sorting and reduction. In order to avoid this problem it has been proposed to solve the so-called core of the problem: a Knapsack Problem defined on a small subset of the variables. The exact core cannot, however, be identified before KP is solved to optimality, thus, previous algorithms had to rely on approximate core sizes. In this paper we present an algorithm for KP where the enumerated core size is minimal, and the computational effort for sorting and reduction also is limited according to a hierarchy. The algorithm is based on a dynamic programming approach, where the core size is extended by need, and the sorting and reduction is performed in a similar “lazy” way. Computational experiments are presented for several commonly occurring types of data instances. Experience from these tests indicate that the presented approach outperforms any known algorithm for KP...

Rollout Algorithms for Combinatorial Optimization

Abstract: We consider the approximate solution of discrete optimization problems using procedures that are capable of magnifying the effectiveness of any given heuristic algorithm through sequential application. In particular, we embed the problem within a dynamic programming framework, and we introduce several types of rollout algorithms, which are related to notions of policy iteration. We provide conditions guaranteeing that the rollout algorithm improves the performance of the original heuristic algorithm. The method is illustrated in the context of a machine maintenance and repair problem.

Semi-Automatic Linear Feature Extraction by Dynamic Programming and LSB-Snakes

Abstract: This paper deals with semi-automatic linear feature extraction from digital images for GIS data capture, where the identification task is pe$ormed manually on a single image, while a special automatic digital module performs the high precision feature tracking in two-dimensional (2-0) image space or even three-dimensional (3-0) object space. A human operator identifies the object from an on-screen display of a digital image, selects the particular class this object belongs to, and provides a very few coarsely distributed seed points. subseq;ently, with th;?sk seed as an approximation of the ~osition and sham the linear feature will be extracted automatically by either a dynamic programming approach or by LSB-S~~~~S [Least-Squares E-spline Snakes). With dynamic programming, the optimization problem is set up as a discrete multistage decision process and is solved by a "timedelayed" algorithm. It ensures global optimality, is numerically stable, and allows for hard constraints to be enforced on the solution. In the least-squares approach, we combine three types of observation equations, one radiometric, formulating the matching of a generic object model with image data, and two that express the internal geometric constraints of a curve and the location of operator-given seed points. The solution is obtained by solving a pair of independent normal equations to estimate the parameters of the spline curve. Both techniques can be used in a monoplotting mode, which combines one image with its underlying DTM. The LSB-S~~~~S approach is also implemented in a multi-image mode, which uses multiple images simultaneously and provides for a robust and mathematically sound full 3D approach. These techniques are not restricted to aerial images. They can be applied to satellite and close-range images as well. The issues related to the mathematical modeling of the proposed methods are discussed and experimental results are shown in this paper too.

Approximate dynamic programming for sensor management

Abstract: This paper studies the problem of dynamic scheduling of multi-mode sensor resources for the problem of classification of multiple unknown objects. Because of the uncertain nature of the object types, the problem is formulated as a partially observed Markov decision problem with a large state space. The paper describes a hierarchical algorithm approach for efficient solution of sensor scheduling problems with large numbers of objects, based on a combination of stochastic dynamic programming and nondifferentiable optimization techniques. The algorithm is illustrated with an application involving classification of 10,000 unknown objects.

A neuro-dynamic programming approach to retailer inventory management

Abstract: We discuss an application of neuro-dynamic programming techniques to the optimization of retailer inventory systems. We describe a specific case study involving a model with thirty-three state variables. The enormity of this state space renders classical algorithms of dynamic programming inapplicable. We compare the performance of solutions generated by neuro-dynamic programming algorithms to that delivered by optimized s-type ("order-up-to") policies. We are able to generate control strategies substantially superior, reducing inventory costs by approximately ten percent.

Dynamic Programming Strategies for the Traveling Salesman Problem with Time Window and Precedence Constraints

Abstract: The Traveling Salesman Problem with Time Window and Precedence Constraints (TSP-TWPC) is to find an Hamiltonian tour of minimum cost in a graph G = (X, A) of n vertices, starting at vertex 1, visiting each vertex i ∈ X during its time window and after having visited every vertex that must precede i, and returning to vertex 1 The TSP-TWPC is known to be NP-hard and has applications in many sequencing and distribution problems In this paper we describe an exact algorithm to solve the problem that is based on dynamic programming and makes use of bounding functions to reduce the state space graph These functions are obtained by means of a new technique that is a generalization of the “State Space Relaxation” for dynamic programming introduced by Christofides et al (Christofides, N, A Mingozzi, P Toth 1981b State space relaxation for the computation of bounds to routing problems Networks 11 145–164) Computational results are given for randomly generated test problems

Capital theory, equilibrium analysis, and recursive utility

Abstract: List of Examples. Preface. Part I: The Recursive Utility Approach: 1. Introduction. 2. What is a Recursive Utility Function? 3. Why Study Recursive Utility? 3.1. The Long Run Incidence of Capital Taxation. The Tax Model. Tax Incidence with the TAS Specification. Tax Incidence With the Epstein-Hynes Utility Specification. 3.2. The Impatience Problem. The Impatience Problem with an Epstein-Hynes Utility Function. 4. Recursive Utility and Commodity Spaces. 4.1 Diminishing Returns and Bounded Growth. 4.2 Nondecreasing Returns and Sustained Growth. Growth and Exogenous Technical Progress. Endogenous Growth Models. 4.3. Order Structures. Weak Separability of the Future from the Present. Partial Orders on the Commodity Space. 5. Conclusion. Part II: Commodity and Price Spaces: 1. Introduction. 2. Commodity Spaces. 2.1. Order Properties. Free Disposal. 2.2 Topological Properties. Metric Spaces. Continuity. Compactness, Product Spaces, and the Tychonoff. Theorem. Connectedness. 2.3. Linear Topologies. Order Convergence. Contraction Mapping Theorems. 3. Commodity Price Dualities. 3.1. Duals and Hyperplanes. 3.2. Hahn-Banach Theorems. 3.3. Dual Pairs and Weak Topologies. 3.4. Order Duals. 4. Conclusion. Part III: Representation of Recursive Preferences: 1. Introduction. 2. Preference Orders and Utility Theory. 3. Recursive Utility: The Koopmans Axioms. 3.1. The Axioms. 3.2. Biconvergence. 3.3. Recursive Preferences and Myopia. 4. Impatience, Discounting and Myopia. 4.1. Impatience and Time Perspective. 4.2. Myopia and the Continuity Axiom. 4.3. The Norm of Marginal Impatience Conditions. 5. Recursive Utility: The Aggregator. 5.1. Basic Properties of the Aggregator. 5.2. The Existence of Recursive Utility. 5.3. Aggregators Bounded From Below. 5.4. Unbounded Aggregators. 6. Conclusion. Part IV: Existence and Characterization of Optimal Paths: 1. Introduction. 2. Fundamentals of Existence Theory. 2.1. A Simple Capital Accumulation Model. 2.2. The Weierstrass Theorem. 2.3. One-Sector TAS Existence Theory. 2.4. Extended Utilitarianism. 3. Multisector Capital Accumulation Model. 3.1. The von Neumann and Malinvaud Models. 3.2. The Feasible Correspondence. 4. The Existence and Sensitivity of Optimal Paths. 4.1. The Maximum Theorem. 4.2. Optimal Paths. 5. Recursive Dynamic Programming. 5.1. Dynamic Programming with TAS Utility. 5.2. Recursive Utility and Multisector Models. 5.3. Dynamic Programming and Extended Utilitarianism. 6. Characterization of Optimal Paths. 6.1. No-Arbitrage Conditions. 6.2. Complete Characterization of Optimal Paths. 7. Conclusion. Part V: 1. Introduction. 2. One-Sector Models. 2.1. Stationary States in One-Sector Models. 2.2. Monotonicity and Turnpikes in TAS Models. 2.3. Monotonocity and Turnpikes in Recursive Models. 2.4. Growing Economies. 3. Steady States in Multisectoral Models. 3.1. Stationary Optimal Programs for Additive Utility. 3.2. Stationary Optimal Programs for Recursive Utility. 4. Stability of Multisectoral Models. 4.1. The Undiscounted Model. 4.2. The Visit Lemma. 4.3. Uniqueness of Steady States. 4.4. Local and Global Stability. 5. Cycles and Chaos in Optimal Growth. 5.1. The Existence of Cycles. 5.2. Chaotic Dynamics. 6. Conclusion. Part VI: Equivalence Principles and Dynamic Equilibria: 1. Introduction. 2. Equivalence Principles for One-Sector Models. 2.1. The Perfect Foresight Equivalence Theorem. Perfect Foresight Competitive Equilibrium. The PFCE Equivalence Principle. 2.2. The Fisher Equivalence Theorem. 2.3. The Equivalence Theorem and the Transversality Condition. 2.4. Recursive Competitive Equilibrium and Recursive Utility. 3. Multisector Equivalence Principles. 3.1. The Portfolio Equilibrium Condition. 3.2. The Two-Sector Model's Equivalence Theorem. The Household Sector. The Production Sector. The Transformation Function. Perfect Foresight Equilibrium. The Optimal Growth Problem. The Equivalence Theorem. 3.3. Dynamics and The Two-Sector Model's Equivalence Theorem. 4. The Transversality Condition and the Hahn Problem. 5. Conclusion. Part VII: Comparative Dynamics: 1. Introduction. 2. The Reduced-Form TAS Model. 2.1. Comparative Dynamics. 2.2. Comparative Dynamics for Oscillating Programs. 2.3. Comparative Dynamics and Capital Income Tax Reform. 3. A Primer of Lattice Programming. 3.1. More About Lattices. 3.2. An Introduction to Monotone Comparative Statistics. 3.3. Topkis's Theorems. 4. Lattice Programming and the Reduced-Form TAS Model. 4.1. The Monotonicity of Optimal Capital Policy Function. 4.2. The Capital Deepening Theorem. 5. Recursive Utility Models. 5.1. Recursive Utility, Monotonicity and Lattice Programming. 5.2. Increasing Impatience and Recursive Utility. 5.3. Capital Deepening and Recursive Utility. 6. Conclusion. Part VIII: Dynamic Competitive Equilibrium: 1. Introduction 2. Dynamic Economies. 2.1. Existence of Pareto Optima. 3. The Core. 3.1. Existence of Core Allocations. 3.2. Edgeworth Equilibria. 4. The Core and Competitive Equilibrium. 4.1. Core Equivalence. 4.2. The Welfare Theorems. 4.3. Representation of Equilibrium as Welfare Maximum. 5. Models with Very Heterogeneous Discounting. 5.1. Core Equivalence with Heterogeneous Discounting. 5.2. Specialization to Recursive Utility. 6. Conclusion. References. Index.

Mathematical Programming for Industrial Engineers

Abstract: Linear programming integer programming graph theory and networks dynamic programming nonlinear programming multiobjective programming stochastic programming heuristic methods.

Dynamic Economics: Optimization by the Lagrange Method

Abstract: This work provides a unified and simple treatment of dynamic economics using dynamic optimization as the main theme, and the method of Lagrange multipliers to solve dynamic economic problems. The author presents the optimization framework for dynamic economics in order that readers can understand the approach and use it as they see fit. Instead of using dynamic programming, the author chooses instead to use the method of Lagrange multipliers in the analysis of dynamic optimization because it is easier and more efficient than dynamic programming, and allows readers to understand the substance of dynamic economics better. The author treats a number of topics in economics, including economic growth, macroeconomics, microeconomics, finance and dynamic games. The book also teaches by examples, using concepts to solve simple problems; it then moves to general propositions.

How to Dynamically Merge Markov Decision Processes

Abstract: We are frequently called upon to perform multiple tasks that compete for our attention and resource. Often we know the optimal solution to each task in isolation; in this paper, we describe how this knowledge can be exploited to efficiently find good solutions for doing the tasks in parallel. We formulate this problem as that of dynamically merging multiple Markov decision processes (MDPs) into a composite MDP, and present a new theoretically-sound dynamic programming algorithm for finding an optimal policy for the composite MDP. We analyze various aspects of our algorithm and illustrate its use on a simple merging problem.

Temporal Dierences-Based Policy Iteration and Applications in Neuro-Dynamic Programming 1

Abstract: We introduce a new policy iteration method for dynamic programming problems with discounted and undiscounted cost. The method is based on the notion of temporal dierences, and is primarily geared to the case of large and complex problems where the use of approximations is essential. We develop the theory of the method without approximation, we describe how to embed it within a neuro-dynamic programming/reinforcement learning context where feature-based approximation architectures are used, we relate it to TD( ) methods, and we illustrate its use in the training of a tetris playing program.

An optimization-based algorithm for scheduling hydrothermal power systems with cascaded reservoirs and discrete hydro constraints

Abstract: An optimization-based algorithm is presented for the short-term scheduling of hydrothermal power systems using the Lagrangian relaxation technique. This paper concentrates on the solution methodology for hydro subproblems with cascaded reservoirs and discrete hydro constraints. Continuous reservoir dynamics and constraints, discontinuous operating regions, discrete operating states and hydraulic coupling of cascaded reservoirs are considered in an integrated fashion. The key idea is to substitute out the reservoir dynamics and to relax the reservoir level constraints by using another set of multipliers, making a hydro subproblem unit-wise and stage-wise decomposable. The optimal generation level for each operating state at each hour can be obtained simply by minimizing a single variable function. Dynamic programming is then applied to optimize the operating states across the planning horizon with a small number of well structured transitions. A modified subgradient algorithm is used to update multipliers. After the dual problem converges, the feasible solution to the hydropower subsystem is obtained by using a network flow algorithm, with operating states obtained in the dual solutions, and possibly adjusted by heuristics. Numerical testing based on practical system data sets show that this method is efficient and effective for dealing with hydrothermal power systems with cascaded reservoirs and discrete hydroelectric constraints.

A genetic algorithm for multiple molecular sequence alignment

Abstract: Motivation: Multiple molecular sequence alignment is among the most important and most challenging tasks in computational biology. The currently used alignment techniques are characterized by great computational complexity, which prevents their wider use. This research is aimed at developing a new technique for efficient multiple sequence alignment. Approach: The new method is based on genetic algorithms. Genetic algorithms are stochastic approaches for efficient and robust searching. By converting biomolecular sequence alignment into a problem of searching for optimal or near-optimal points in an 'alignment space', a genetic algorithm can be used to find good alignments very efficiently. Results: Experiments on real data sets have shown that the average computing time of this technique may be two or three orders lower than that of a technique based on pairwise dynamic programming, while the alignment qualities are very similar. Availability: A C program on UNIX has been written to implement the technique. It is available on request from the authors.

Risk-Sensitive Control of Finite State Machines on an Infinite Horizon I

Abstract: In this paper we consider robust and risk-sensitive control of discrete time finite state systems on an infinite horizon. The solution of the state feedback robust control problem is characterized in terms of the value of an average cost dynamic game. The risk-sensitive stochastic optimal control problem is solved using the policy iteration algorithm, and the optimal rate is expressed in terms of the value of a stochastic dynamic game with average cost per unit time criterion. By taking a small noise limit, a deterministic dynamic game which is closely related to the robust control problem is obtained.

An information theoretic approach to observer path design for bearings-only tracking

Abstract: Open-loop control strategies, via information theoretic criteria, for the design of optimal observer trajectories in the bearings-only tracking problem are presented. The aim is to obtain tight bounds on the location and velocity of a single target through own ship maneuvers. In this paper, optimal paths are derived by maximizing the mutual information between the measurement sequence and the final target state or the entire target trajectory. Optimization techniques, such as dynamic programming and enumeration with optimal pruning are derived.

Training strategies for critic and action neural networks in dual heuristic programming method

Abstract: This paper discusses strategies for and details of training procedures for the dual heuristic programming methodology. This and other approximate dynamic programming approaches have been discussed in the literature, all being members of the adaptive critic design family. It suggests and investigates several alternative procedures and compares their performance with respect to convergence speed and quality of resulting controller design. A modification is to introduce a real copy of the criticNN (criticNN 2) for making the "desired output" calculations, and this criticNN 2 is trained differently than is criticNN 1. The idea is to provide the "desired outputs" from a stable platform during an epoch while adapting the criticNN 1. Then at the end of the epoch, criticNN 2 is made identical to the then-current adapted state of criticNN 1, and a new epoch starts. In this way, both the criticNN 1 and the actionNN can be simultaneously trained online during each epoch, with a faster overall convergence than the older approach. The measures used suggest that a "better" controller design (the actionNN) results.

A parallel genetic algorithm approach to solving the unit commitment problem: implementation on the transputer networks

Abstract: Through a constraint handling technique, this paper proposes a parallel genetic algorithm (GA) approach to solving the thermal unit commitment (UC) problem. The developed algorithm is implemented on an eight-processor transputer network, processors of which are arranged in master-slave and dual-direction ring structures, respectively. The proposed approach has been tested on a 38-unit thermal power system over a 24-hour period. Speed-up and efficiency for each topology with different number of processor are compared to those of the sequential GA approach. The proposed topology of dual-direction ring is shown to be well amenable to parallel implementation of the GA for the UC problem.

Variable splitting applied to modelling of start-up costs in short term hydro generation scheduling

Abstract: In this paper, a mixed-integer hydroelectric power model for short-term power system planning is discussed. The purpose of the model is to represent the generation characteristics of the hydropower plants accurately. In the modelling part, the paper focuses on two requests stated from power producers in an earlier study. First, it is desirable to operate on points with good efficiency. Second, start-ups and shut-downs of the units should not be carried out too often, since this is related to a cost. The problem is decomposed into a subproblem for each hydropower plant by relaxing the power balance equation and using variable splitting. The advantage of this relaxation is that network programming and dynamic programming methods can be used, which have proved to work well for power system planning issues. The relaxation used in this work has, as far as the authors know, not been applied in the field before. The master problem is solved by a subgradient technique. Finally, a numerical example from a part of the Swedish power system illustrates the technique.

Joint Minimization of Code and Data for Synchronous DataflowPrograms

Abstract: In this paper, we formally develop techniques that minimize the memory requirements of a target program when synthesizing software from dataflow descriptions of multirate signal processing algorithms. The dataflow programming model that we consider is the synchronous dataflow (SDF) model [21], which has been used heavily in DSP design environments over the past several years. We first focus on the restricted class of well-ordered SDF graphs. We show that while extremely efficient techniques exist for constructing minimum code size schedules for well-ordered graphs, the number of distinct minimum code size schedules increases combinatorially with the number of vertices in the input SDF graph, and these different schedules can have vastly different data memory requirements. We develop a dynamic programming algorithm that computes the schedule that minimizes the data memory requirement from among the schedules that minimize code size, and we show that the time complexity of this algorithm is cubic in the number of vertices in the given well-ordered SDF graph. We present several extensions to this dynamic programming technique to more general scheduling problems, and we present a heuristic that often computes near-optimal schedules with quadratic time complexity. We then show that finding optimal solutions for arbitrary acyclic graphs is NP-complete, and present heuristic techniques that jointly minimize code and data size requirements. We present a practical example and simulation data that demonstrate the effectiveness of these techniques.