Showing papers on "Dynamic programming published in 1972"

Minimum-Time Routing as an N-Stage Decision Process

Abstract: In this paper the problem of determining minimum-time ship routes is expressed as a discrete decision process in stochastic conditions, and is solved by dynamic programming. Some models of such processes are considered. Their applicability depends on the amount of available meteorological information. The salient features of the computer programs, operating experimentally aboard a motorship in normal service, are illustrated and example trajectories are presented.

A mixed optimization technique for the generalized machine replacement problem

Abstract: : A mixed optimization technique for optimal machine replacement is presented which allows much more flexibility than previous models. Optimal purchase, maintenance, and sale of a given machine between any two given points in time is treated as a sub-problem, which one may choose to solve via control theory, dynamic programming, or practical engineering considerations. (A control theory formulation is used in the paper as an illustration.) These sub-problem solutions are then incorporated into a Wagner-Whitin formulation for solution of the full problem. The technique is particularly useful for problems with such asymmetries as an existing initial machine or uneven technological change. (Author)

An efficient algorithm for optimum decomposition of recycle systems

Abstract: A method is developed for decomposition of a recycle process so as to minimize the summation of weighting factors for variables torn. This procedure is based on application of dynamic programming to a state space representing combinations of cycles opened. The resulting algorithm is well suited to machine implementation and is more efficient for large problems than alternative procedures that also guarantee decomposition.

Optimal synchronization of traffic signal networks by dynamic programming

Abstract: THIS PAPER PRESENTS A SYNCHRONIZATION METHOD FOR DETERMINATION OF OPTIMAL OFFSETS IN ROAD TRAFFIC NETWORKS CONTROLLED BY FIXED-TIME SIGNALS. THE METHOD IS BASED ON DYNAMIC PROGRAMMING AND PROVIDES AN OPTIMAL SOLUTION, INDEPENDENT OF NETWORK LAYOUT, IN A FINITE NUMBER OF COMPUTATION STEPS. THE MATHEMATICAL MODEL OF THE SYNCHRONIZATION PROBLEM IS INTRODUCED FIRST. DEFINITIONS OF THE SYSTEM'S INDEPENDENT VARIABLES ARE GIVEN AND THE EQUATIONS CHARACTERIZING THE CONSTRAINTS IMPOSED ON THE OFFSET ACROSS-VARIABLES ARE FORMULATED. THE DEPENDENT VARIABLES OF THE SYSTEM ARE THE COST FUNCTIONS ASSOCIATED WITH EACH LINK OF THE TRAFFIC NETWORK. THE OPTIMIZATION TARGET IS A MINIMIZED ECONOMIC OBJECTIVE FUNCTION COMPRISING THE INDIVIDUAL LINK FUNCTIONS AND POSSIBLY IN- CLUDING DELAY TIMES AS WELL AS STOPS. THE PROBLEM IS NONLIN- EAR (OWING TO THE CHARACTER OF THE LINK COST FUNCTIONS) AND CONTAINS INTEGER VARIABLES IN THE CIRCUIT CONSTRAINT EQUA- TIONS. THE ALGORITHM FOR ITS SOLUTION IS BASED ON PARTIAL MINIMIZATIONS CONCLUDED BY DETERMINING A FUNDAMENTAL SET OF OPTIMAL OFFSETS AS WELL AS THE OPTIMAL VALUE OF THE OBJECT- IVE FUNCTION. IN ORDER TO OVERCOME THE "CURSE OF DIMENSION- ALITY" INHERENT IN MULTIVARIABLE PROBLEMS, A DECISION-TREE ALGORITHM IS EMPLOYED FOR OBTAINING A MINIMIZATION PLAN WHICH IS OPTIMAL IN TERMS OF THE COMPUTATIONAL EFFORT INVESTED. THE AIM, IN THIS CASE, IS TO MINIMIZE THE REQUIRED COMPUTER STORAGE CAPACITY AND NUMBER OF OPERATIONS.

Minimum-energy control of a class of electrically driven vehicles

Abstract: A minimum-energy controller is designed and built for a class of electrically driven vehicles according to the theoretical concepts determined by the application of modern control theory. Theoretical results are obtained by making several justifiable assumptions in the dynamical equations of the system and solving the resulting stochastic optimal control problem by Bellman's dynamic programming technique. Several practical and economical considerations are taken into account for the mechanization of the minimum-energy control law.

On the solution of some minimax problems

Abstract: In dynamic minimax and stochastic optimization problems frequently one is forced to use a suboptimal controller since the computation and implementation of the optimal controller based on dynamic programming is impractical in many cases. In this paper we study the performance of some suboptimal controllers in relation to the performance of the optimal feedback controller and the optimal open-loop controller. Attention is focused on some classes of, so called, open-loop-feed-back controllers. It is shown under quite general assumptions that these open-loop-feedback controllers perform at least as well as the optimal open-loop controller. The results are developed for general minimax problems with perfect and imperfect state information. In the latter case the open-loop-feedback controller mkes use of an estimator which is required to perform at least as well as a pure predictor in order for the results to hold. Some of the results presented have stochastic counterparts.

Representation theorems for equivalent optimization problems

Abstract: In conjunction with the problem of transforming a given optimization problem into a form from which the functional equations of dynamic programming are obtainable, Karp and Held (1967) made clear the relation between a certain class of decision processes and dynamic programming from the view point of automata theory. This paper also follows the line of Karp and Held, and presents a number of new concepts. First we assume that a given optimization problem is discrete and deterministic: it is given in the form of discrete decision process (ddp). Then we define six classes of decision processes: sdp (sequential decision process), msdp (monotone sdp), smsdp (strictly monotone sdp), pmsdp (positively monotone sdp), ap (additive process), and lmsdp (loop-free msdp). The sdp is considered as a general model of a decision process with finite states. The msdp is a subclass of sdp's from which the functional equations of dynamic programming are obtainable. The smsdp, pmsdp, ap, and lmsdp are subclasses of msdp's, which have simpler structures than that of msdp. In fact, simpler solution methods for solving the resulting functional equations are available for these subclasses. Two types of representation theorems are first proved for each class of decision processes: one is the w (weak)-representation theorem which is a necessary and sufficient condition for a given ddp to be realized by a decision process of the specific class in the sense that both have the same set of optimal policies, and the other is the s (strong)-representation theorem, which assumes the coincidence of cost value for each feasible policy in addition to the above condition. Based on the w -representation theorems, various properties of sets of optimal policies are investigated for each class. In particular, it is shown that although sets of optimal policies of sdp and msdp are not closed under most of operations, they are closed for smsdp, pmsdp, ap, and lmsdp. In fact, a set of policies can be a set of optimal policies of an smsdp, pmsdp, or ap if and only if it is regular (i.e., accepted by a finite automaton). For an lmsdp, a set can be a set of optimal policies if and only if it is finite.

Dynamic Programming Training Period for an MSE Adaptive Equalizer

Abstract: This paper concerns itself with the design of an algorithm that will shorten the training period and adaptation time of an adaptive equalizer. Time-invariant or slowly varying channels with white additive Gaussian noise are considered. An adaptive equalizer in the form of a nonrecursive transversal filter reduces the intersymbol interference. The training period consists of the transmission of isolated pulses between which the equalizer is adjusted. The algorithm uses a minimum mean-squared error criterion with a variable step size on each iteration. A fixed number of iterations is allowed for the error to be minimized. A constraint related to the average excess mean-squared error is included, and the set of step sizes is determined by invoking the principle of optimality in dynamic programming. The resulting algorithm is compared to the popular fixed step-size algorithm. Predicted and experimental results are given. A fairly well conditioned and a poorly conditioned channel are considered. Results show that the new algorithm has a faster adaptation time. It is more complex than the fixed step-size algorithm, but for long transversal filters requires little additional computation time.

An innovative approach to compensator design

Abstract: The design is considered of a computer-aided-compensator for a control system from a frequency domain point of view. The design technique developed is based on describing the open loop frequency response by n discrete frequency points which result in n functions of the compensator coefficients. Several of these functions are chosen so that the system specifications are properly portrayed; then mathematical programming is used to improve all of these functions which have values below minimum standards. To do this, several definitions in regard to measuring the performance of a system in the frequency domain are given, e.g., relative stability, relative attenuation, proper phasing, etc. Next, theorems which govern the number of compensator coefficients necessary to make improvements in a certain number of functions are proved. After this a mathematical programming tool for aiding in the solution of the problem is developed. This tool is called the constraint improvement algorithm. Then for applying the constraint improvement algorithm generalized, gradients for the constraints are derived. Finally, the necessary theory is incorporated in a Computer program called CIP (compensator Improvement Program). The practical usefulness of CIP is demonstrated by two large system examples.

Dynamic programming and a max-min problem in the theory of structures

Abstract: A max—min problem in the realm of optimum beam design is formulated and thoroughly investigated from a dynamic programming point of view. It is shown that the conditions of optimality can be directly derived from the Hamilton—Jacobi—Bellman equation of the process. The classical Euler—Lagrange equations for the beam are derived from the fundamental partial differential equation. It is shown that the conditions of optimality associated with the minimum operation are local expressions of the theorem of Castigliano. An analytical solution for the unconstrained optimum cantilever laying on elastic foundation is presented, and a method of successive approximations consisting in a stable, two-sweep iterative procedure, is developed. Numerical examples are given.

A dynamic programming—integer programming algorithm for allocating touristic investments

Abstract: In a companion paper (1) a general mathematical model for the allocation of touristic investments was developed. In this paper a solution methodology for the model is developed based on the principles of dynamic programming. At each stage of the dynamic program an integer program is solved to limit the range of values of the state variable which must explicitly be considered. The algorithm is illustrated through an example, and the advantages of the solution procedure are explained by considering the solution as a base for the strategic decision making in the touristic sector.

Computational techniques for optimizing systems with standby redundancy

Abstract: : Three methods are used to solve the following problem: for P, a positive constant, maximize (P. Reliability -cost) of a system with standby redundancy. The results show that a method which rounds a nonintegar solution to the nearest integar solution can lead to tremendous mistakes. However, neither a well known dynamic programming algorithm nor a previously developed branch and bound technique are able to solve large size problems. The solution of problems of large dimension thus requires the use of the noninteger solution of the first method to limit the number of possible solutions when using either the dynamic programming algorithm or a modified branch and bound technique. With this assitance, the branch and bound technique is able to solve large problems in a short amount of computational time. (Author)

Parallel processing algorithms for modal trajectory estimation

Abstract: : For modal trajectory state estimation, i.e., estimation of the maximum likelihood trajectory in state space, the problem can be solved using the idea of dynamic programming. Since there are a number of parallel operations that occur in the evaluation of the dynamic programming recursive formula, the use of a parallel computer could greatly reduce the computer time and memory required for obtaining the modal trajectory estimate. The purpose of the paper is to discuss the modal trajectory estimation method and how various algorithms for implementing dynamic programming in a parallel processor can be used to reduce the computational burden. In particular, the following algorithms for implementing dynamic programming in parallel processors are examined: Parallel States Algorithm; Parallel Noises Algorithm; and Parallel States and Stages Algorithm. (Author)

Dynamic Programming Training Period for an MSE Adaptive Equalizer

Abstract: This paper concerns itself with the design of an algorithm that wilI shotten the training period and adaptation time of an adaptive equalizer. Time-invariant or slowly varying channels with white additive Gaussian noise are considered. An adaptive equalizer in the form of a nonrecursive transversal filter reduces the intersymbol interference. The training period consists of the transmission of isolated pulses be- tween which the equalizer is adjusted. The algorithm uses a minimum mean-squared error criterion with a variable step size on each iteration. A fixed number of iterations is allowed for the error to be minimized. A constraint related to the average excess mean-squared error is included, and the set of step sizes is determined by invoking the principle of optimality in dynamic programming. The resulting algorithm is compared to the popular fixed step-size algorithm. Predicted and experimental results are given. A fairly well conditioned and a poorly conditioned channel are considered. Results show that the new algorithm has a faster adaptation time. It is more complex than the fixed step-size algorithm, but for long transversal filters requires little additional kmputation time.

An approximate method for the synthesis of optimal control of distributed systems

Abstract: An approximate method is presented for the synthesis of an optimal control of distributed parameter systems, based on a combination of the ideas of Lyapunov's direct method and those of Bellman's successive approximations in policy space. The method is such that the approximate equation is improved after each iteration in the sense of the performance index being used.

Deterministic Processes of Dynamic Programming with Additional Constraints

Abstract: In certain practical situations we encounter problems of dynamic programming with additional constraints on the optimal trajectories that have an integral character. For example, in detection problems [1] such constraints can be given in the form of the admissible number of false alarms, which in the case of a natural formulation of the problem must increase with the observation time. In production problems such a constraint may appear as a result of the fact that for the process one uses a certain item (fuel, raw material) whose intensity of arrival is assigned, with only fairly small deviations being permitted in the total consumption of this item with respect to the total amount supplied.

Techniques for generating highly reliable redundant systems.

Abstract: A simple heuristic algorithm for designing highly reliable modularly redundant computer systems under complexity constraints is presented. The technique, which produces near optimal solutions, is intuitively appealing and easy to apply. The algorithms performance is shown to compare very well with the optimal solution obtained via a computerized model for dynamic programming.

An introduction to structural optimization problems

Abstract: The object of the research described in this thesis is to examine the possibilities of developing analytical and computational procedures for a class of structural optimization problems in the presence of behaviour and side constraints. These are essentially optimal control problems based on the maximum principle of Pontryagin and dynamic programming formalism of Bellman. They are characterised by inequality constraints on the state and control variables giving rise to systems of highly complex differential equations which present formidable difficulties both in the construction of the appropriate boundary conditions and subsequent development of solution procedures for these boundary value problems. Therefore an alternative approach is used whereby the problem is discretised leading to a non-linear programming approximation. The associated non-linear programs are characterised by non-analytic "black box" type representations for the behaviour constraints. The solutions are based on a "steepest descent–alternate step" mode of travel in design space. [Continues.]

More on a Class of Optimal Search Problems

Abstract: : Optimal strategies are investigated for a class of one-dimensional search processes in which the objective is to find a point which is near, but not beyond, a boundary of uncertain location. Problems of this type are encountered in the analysis of mining operations. Upper and lower bounds for the optimal expected payoff are derived, and the optimal search strategies are described explicitly for a large subclass of these processes. Results are obtained by formulating the search as a multistage decision process and using a dynamic programming approach.

A computational successive improvement scheme for adaptive optimal control processes

Abstract: To computationally solve an adaptive optimal control problem by means of conventional dynamic programming, a backward recursion must be used with an optimal value and optimal control determined for all conceivable prior information patterns and prior control histories. Consequently, almost all problems are beyond the capability of even large computers.

Edge Detection Using Heurisitic Search Methods.

Abstract: : The paper presents a method for detecting edges in a digitized picture. The problem of edge detection is reduced to the problem of finding an optimal path in a weighted graph. The properties of the edge are embeded in the structure of the graph. Graph searching techniques are then used to find the optimal solution. Usually, the optimality of the solution is not important, and several heuristics can be introduced to reduce the search. Because of the global approach, this method has great flexibility. Some experimental results are given which show the performance of this method with noisy pictures. (Author)

Optimal load scheduling of hydroelectric power stations.

Abstract: The work presented in this thesis concerns the optimisation of the hour-to-hour generation schedule (over a scheduling interval of 24 hours) of the generating stations in an all-hydro electric - system. The day-to-day system storage policy is assumed known from the long-term schedule. System load demand is first treated as deterministic. Later, attempts are made to treat it as stochastic. Throughout the text, system transmission losses are represented by means of the B-coefficient loss formula. Chapter 1 presents brief discussions on various topics which may play - a part in the optimization process. In particular, the concept of frequency control and the distinction between long-term and daily scheduling are outlined. The optimization objective is then formulated and some of the known methods of obtaining the solution are briefly discussed. In chapter 2, the optimization problem is first formulated . as that of minimizing the instantaneous total power drawn from the system subject to satisfying a . predetermined time-varying load demand.This is analogous to the minimization of the instantaneous total cost rate in thermal systems. Station characteristic curves are approximated by straight line segments and head variations are assumed negligible. The optimising - conditions are derived using the Lagrange multiplier technique. A small system consisting of 4 stations is studied, where optimisation is achieved by means of an analogue computer. By introducing weighting factors in the objective function, the storage policy can be satisfied. (Analysis carried out in chapter 3 later reveals that this is the correct measure which must be taken to meet the storage policy). Problems do arise, however, when - there are multi-unit stations present in the system. Due to the non-monotonic nature . of the incremental Output curves' of such stations, a. number of solutions would satisfy the optimising conditions. The optimum solution can only be established after testing each possible solution, a prohibitive task for large systems. Approximating the incremental output curves of such stations may create undesirable errors. Another weakness of the method is that it cannot readily take into account the effects of head variations. The application of Pontryagin's Maximum Principle is attempted in chapter 3. It is shown that, for systems with fixed— head stations, the optimising conditions are exactly identical to that derived using the Lagrange multiplier technique with weightings being introduced in the objective function. Unfortunately, those problems associated with the presence of multi—unit stations are still unsolved. Under ideal system conditions, it is shown that optimum generation schedule would result if all stations; except the frequency control station, were step loaded. Step loading mode is defined as an operating mode where the instantaneous station discharge rate is only allowed to take one of two discrete values with stepwise transitions. The analysis indicates that the switching instants do not play any part, except they must be so chosen that the output of the frequency control station is kept within its limits. When step loading technique is applied to non—ideal systems, the optimisation. becomes that of searching for the optimum switching instants of the step loaded stations. This is discussed in chapter 4. For practical reasons, only two step loading modes are assumed admissible. A system consisting of 18 stations is studied. Experimental results indicate that the optimum step loading modes can be established within - a reasonably short computation time. One striking feature of the step loading technique is that it can handle the optimal scheduling problem where the load demand is treated as being stochastic. The effects of head variations can be estimated fairly accurately. The application of the principle of dynamic programming is discussed in chapter 5. Due to the multi-dimensional nature of the problem, this principle must be combined with the relaxation principle to minimise the computation time and computer storage requirements. A good initial guess for the optimum schedule may be obtained using step loading technique. The step loading modes are then modified by applying the combination of dynamic programming and relaxaxtion principles. Finally, chapter 6 outlines the general conclusions drawn from the work presented in the preceeding chapters. The possibility. of applying Pontryagin's Maximum Principle to obtain the long-term storage policy is also studied. Related future research areas are suggested. To the author's best knowledge, the materials presented in chapters 3, 4 and 5 are original. Chapter 2 is based on the work of Chandler Jr. and Gabrielle, where the mathematical error found in their text has been corrected and the weighting procedure, among other materials has been included.