Showing papers on "Dynamic programming published in 1984"
TL;DR: The algorithm to be developed is essentially identical to one presented by Vintsyuk and later by Bridle and Brown, but the notation and the presentation have been clarified and the computational expenditure per word is independent of the number of words in the input string.
Abstract: This paper is of tutorial nature and describes a one-stage dynamic programming algorithm for file problem of connected word recognition. The algorithm to be developed is essentially identical to one presented by Vintsyuk [1] and later by Bridle and Brown [2] ; but the notation and the presentation have been clarified. The derivation used for optimally time synchronizing a test pattern, consisting of a sequence of connected words, is straightforward and simple in comparison with other approaches decomposing the pattern matching problem into several levels. The approach presented relies basically on parameterizing the time warping path by a single index and on exploiting certain path constraints both in the word interior and at the word boundaries. The resulting algorithm turns out to be significantly more efficient than those proposed by Sakoe [3] as well as Myers and Rabiner [4], while providing the same accuracy in estimating the best possible matching string. Its most important feature is that the computational expenditure per word is independent of the number of words in the input string. Thus, it is well suited for recognizing comparatively long word sequences and for real-time operation. Furthermore, there is no need to specify the maximum number of words in the input string. The practical implementation of the algorithm is discussed; it requires no heuristic rules and no overhead. The algorithm can be modified to deal with syntactic constraints in terms of a finite state syntax.
364 citations
01 Jan 1984
TL;DR: The convex hull of the solutions of the economic lot-sizing model is given, and an alternative formulation as a simple plant location problem is examined, and here too the convex Hull of solutions is obtained.
Abstract: It is well-known that the economic lot-sizing model is well-solved by dynamic programming. On the other hand, the standard mixed integer programming formulation of this problem leads to a very large duality gap. Here the convex hull of the solutions of the economic lot-sizing model is given. In addition, an alternative formulation as a simple plant location problem is examined, and here too the convex hull of solutions is obtained.
200 citations
114 citations
TL;DR: In this paper, a new algorithm for finding all solutions with objective function values in the neighborhood of the optimum for certain dynamic programming models, including shortest path problems, is presented, which combines the depth-first search with stacking techniques of theoretical computer science and principles from dynamic programming to modify the usual backtracking routine and list all near-optimal policies.
Abstract: This paper presents a new algorithm for finding all solutions with objective function values in the neighborhood of the optimum for certain dynamic programming models, including shortest path problems. The new algorithm combines the depth-first search with stacking techniques of theoretical computer science and principles from dynamic programming to modify the usual backtracking routine and list all near-optimal policies. The resulting procedure is the first practical algorithm for a variety of large problems that are of interest.
85 citations
TL;DR: In this paper, it is shown that the optimal decision is characterized by thresholds as in the decoupled case, however, the thresholds are time-varying and their computation requires the solution of two coupled sets of dynamic programming equations.
Abstract: Two detectors making independent observations must decide when a Markov chain jumps from state 0 to state 1. The decisions are coupled through a common cost function. It is shown that the optimal decision is characterized by thresholds as in the decoupled case. However, the thresholds are time-varying and their computation requires the solution of two coupled sets of dynamic programming equations. A comparison to the decoupled case shows the structure of the coupling.
60 citations
TL;DR: In this article, a mixed approach of depth-first search-dynamic programming (DFLP) and the Ahrens-Finke algorithm was proposed to solve the Subset-Sum Problem (SSP).
Abstract: Given n items, each having a weight wi, and a container of capacity W, the Subset-Sum ProblemSSP is to select a subset of the items whose total weight is closest to, without exceeding, W.
The paper presents a mixed approach depth first search-dynamic programming to the exact solution of the problem. An extensive computational experience is presented, comparing the proposed algorithm with that of Ahrens-Finke, as well as with the Balas-Zemel algorithm for large problems. Both "easy" and "hard" problems with values of n up to 10,000 are considered.
60 citations
01 Jan 1984
TL;DR: This paper deals with negative dynamic programming problems, i.e. discrete time total reward problems with non-positive reward functions, with countable state space, and shows that e-optimal stationary policies exist in general dynamic Programming problems if this is true for the imbedded negative model.
Abstract: This paper deals with negative dynamic programming problems, i.e. discrete time total reward problems with non-positive reward functions, with countable state space. The main question, treated here, is the existence of e-optimal stationary policies. This was motivated by results in /2/, which show that e-optimal stationary policies exist in general dynamic programming problems if this is true for the imbedded negative model.
57 citations
TL;DR: A constructive proof of the existence of optimal policies among all policies under new cumulative average optimality criteria which are more sensitive than the maximization of the spectral radius is given.
Abstract: Previous treatments of multiplicative Markov decision chains eg., Bellman [Bellman, R. 1957. Dynamic Programming. Princeton University Press, Princeton, New Jersey.], Mandl [Mandl, P. 1967. An iterative method for maximizing the characteristic root of positive matrices. Rev. Roumaine Math. Pures Appl.XII 1317--1322.], and Howard and Matheson [Howard, R. A., Matheson, J. E. 1972. Risk-sensitive Markov decision processes. Management Sci.8 356--369.] restricted attention to stationary policies and assumed that all transition matrices are irreducible and aperiodic. They also used a “first term” optimality criterion, namely maximizing the spectral radius of the associated transition matrix. We give a constructive proof of the existence of optimal policies among all policies under new cumulative average optimality criteria which are more sensitive than the maximization of the spectral radius. The algorithm for finding an optimal policy, first searches for a stationary policy with a nonnilpotent transition matrix, provided such a rule exists. Otherwise, the method still finds an optimal policy; though in this case the set of optimal policies usually does not contain a stationary policy! If a stationary policy with a nonnilpotent transition matrix exists, then we develop a policy improvement algorithm which finds a stationary optimal policy.
55 citations
01 Dec 1984
TL;DR: In this paper, the authors consider a dynamic system whose state is governed by a linear stochastic differential equation with time-dependent coefficients, and their objective is to minimize an integral cost which depends upon the evolution of the state and the total variation of the control process.
Abstract: We consider a dynamic system whose state is governed by a linear stochastic differential equation with time-dependent coefficients. The control acts additively on the state of the system. Our objective is to minimize an integral cost which depends upon the evolution of the state and the total variation of the control process. It is proved that the optimal cost is the unique solution of an appropriate free boundary problem in a space-time domain. By using some decomposition arguments, the problems of a two-sided control, i.e. optimal corrections, and the case with constraints on the resources, i.e. finite fuel, can be reduced to a simpler case of only one-sided control, i.e. a monotone follower. These results are applied to solving some examples by the so-called method of similarity solutions.
46 citations
TL;DR: The algorithm for three sequence comparisons, which is based on solving a system of recursive equations, improves upon the efficiency of existing methods and is simplified by appeal to combinatorial considerations.
43 citations
TL;DR: In this paper, a general approach to solve the problem of unit commitment and fossil fuel allocation is presented, which is tested on a system with 60 units and some characteristic features of the optimal solution are discussed.
Abstract: A general approach will be presented to solve the problem of unit commitment and fossil fuel allocation. Multi-fuel units allowed to burn a variable mixture of different fuel types, with fuel dependent input-outpat models are considered in the optimization process. In addition, a variety of contractual limitations and physical constraints may be included. The approach has been tested on a system with 60 units. Results will be presented. Some characteristic features of the optimal solution will be discussed. From these, simplified approaches may be derived in order to solve dedicated problems.
TL;DR: A state-of-the-world decomposition approach is introduced that solves the dynamic probabilistic generation expansion problem using simple static deterministic solution techniques and represents the minimum discounted expected cost generation expansion plan.
Abstract: Dynamics and uncertainty are central to electric utility generation expansion planning, but are difficult to handle explicitly. A state-of-the-world decomposition approach is introduced that solves the dynamic probabilistic generation expansion problem using simple static deterministic solution techniques. The main problem is decomposed into a set of static deterministic problems. Each problem represents a distinct state-of-the-world (i.e., a time and outcome scenario) and is solved individually using a new dynamic programming procedure. The problems are linked through Lagrange multipliers that are determined iteratively and that can be interpreted as “shadow” fixed costs. Consequently, one difficult problem is replaced with many easy ones. The solution obtained represents the minimum discounted expected cost generation expansion plan. It reflects the importance of future conditions in current decisions and the utility's ability to respond to the resolution of uncertainties over time.
TL;DR: In this article, a multiobjective linear quadratic Gaussian control problem with a hierarchical ordering of the individual objective functions is formulated and solved, where the objective functions are ordered in a hierarchical manner.
Abstract: A multiobjective linear quadratic Gaussian control problem with a hierarchical ordering of the individual objective functions is formulated and solved.
01 May 1984
TL;DR: Algorithms for the dynamic programming and transitive closure problems are presented for a linear pipeline of processors that require only a constant number of input/output ports and are optimal in their area and time requirements.
Abstract: : Algorithms for the dynamic programming and transitive closure problems are presented for a linear pipeline of processors. These algorithms require only a constant number of input/output ports and are optimal in their area and time requirements. (Author)
TL;DR: In this paper, the authors show that if the lead time distributions are arbitrary except that they are independent of order size and do not allow orders to cross in time, then each order in an optimal solution will exactly satisfy a consecutive sequence of demands, a natural extension of the classic results by Wagner and Whitin.
Abstract: Optimal solutions for the dynamic lot-sizing problem with deterministic demands but stochastic lead times are "lumpy." If lead time distributions are arbitrary except that they are independent of order size and do not allow orders to cross in time, then each order in an optimal solution will exactly satisfy a consecutive sequence of demands, a natural extension of the classic results by Wagner and Whitin. If, on the other hand, orders can cross in time, then optimal solutions are still "lumpy" in the sense that each order will satisfy a set, not necessarily consecutive, of the demands. An example shows how this characterization can be used to find a solution to a problem where interdependence of lead times is critical. This characterization of optimal solutions facilitates dynamic programming approaches to this problem.
Posted Content•
TL;DR: A maximum principle is derived for dynamic systems with continuous lags, i.e., systems governed by integrodifferential equations, using dynamic programming and the adjoint variables can be provided with useful economic interpretations.
Abstract: This paper derives a maximum principle for dynamic systems with continuous lags, i.e., systems governed by integrodifferential equations, using dynamic programming. As a result, the adjoint variables can be provided with useful economic interpretations.
TL;DR: In this paper, the authors derived a maximum principle for dynamic systems with continuous lags, i.e., systems governed by integrodifferential equations, using dynamic programming, and provided useful economic interpretations.
Abstract: This paper derives a maximum principle for dynamic systems with continuous lags, i.e., systems governed by integrodifferential equations, using dynamic programming. As a result, the adjoint variables can be provided with useful economic interpretations.
TL;DR: A simple approach to solving the optimal static mix problem is introduced here that is unique and has several useful properties and can be extended to handle general capacity and energy-related constraints and other complications.
Abstract: Considerable insight can be gained in electric utility generation expansion planning by solving the optimal static mix problem; that is, by determining the best set of generation technologies under fixed conditions. A simple approach to solving this problem is introduced here that is unique and has several useful properties. The fundamental idea is to treat the static problem as a dynamic program, whose stages are technologies (not time periods) and whose state is cumulative capacity. This approach is fast; it handles unit sizes and capacity constraints trivially; and it can be extended to handle general capacity and energy-related constraints and other complications.
TL;DR: The optimal scheduling or unit commitment of power generation systems to meet a random demand involves the solution of a class of dynamic programming inequalities for the optimal cost and control law, which is studied in terms of a scheduling delay and the relative magnitudes of the costs of different units.
Abstract: The optimal scheduling or unit commitment of power generation systems to meet a random demand involves the solution of a class of dynamic programming inequalities for the optimal cost and control law. We study the behavior of this optimality system in terms of two parameters: (i) a scheduling delay, e.g., the startup time of a generation unit; and (ii) the relative magnitudes of the costs (operating or starting) of different units. In the first case we show that under reasonable assumptions the optimality system has a solution for all values of the delay, and, as the delay approaches zero, that the solutions converge uniformly to those of the corresponding system with no delays. In the second case we show that as the cost of operating or starting a given machine increases relative to the costs of the other machines, there is a point beyond which the expensive machine is not used, except in extreme situations. We give a formula for the relative costs that characterize this point. Moreover, we show that as the relative cost of the expensive machine goes to infinity that the optimal cost of the system including the expensive machine approaches the optimal cost of the system without the machine.
TL;DR: It is proved that the optimal average cost is independent of the initial state and that the cost-to-go functions of dynamic programming are convex, leading to the conclusion that optimal policies are switching policies.
Abstract: An infinite horizon, expected average cost, dynamic routing problem is formulated for a simple failure-prone queueing system, modelled as a continuous time, continuous state controlled stochastic process. We prove that the optimal average cost is independent of the initial state and that the cost-to-go functions of dynamic programming are convex. These results, together with a set of optimality conditions, lead to the conclusion that optimal policies are switching policies, characterized by a set of switching curves (or regions), each curve corresponding to a particular state of the nodes (servers).
TL;DR: This paper investigates the computation of transient-optimal policies in discrete dynamic programming and the concept of superharmonicity is introduced, which provides the linear program to compute the transientvalue-vector and a transient- optimal policy.
Abstract: This paper investigates the computation of transient-optimal policies in discrete dynamic programming. The model, is quite general: it may contain transient as well as nontransient policies. and the transition matrices are not necessarily substochastic.
A functional equation for the so-called transient-value-vector is derived and the concept of superharmonicity is introduced. This concept provides the linear program to compute the transientvalue-vector and a transient-optimal policy.
We also discuss the elimination of suboptimal actions, the solution of problems with additional constraints, and the computation of an efficient policy for a multiple objective dynamic programming problem.
TL;DR: In this article, a non lineaire elliptique equation non-lineaire is proposed, which is a solution unique sous certain hypotheses, and a methode de programmation dynamique.
Abstract: L'etude de problemes de commande optimale stochastique de processus de diffusion controles a la fois par impulsions et de facon continue, et arretes a la frontiere d'une region bornee de R N , conduit par une methode de programmation dynamique, a une equation non lineaire elliptique. On montre que cette equation a une solution unique sous certaines hypotheses
TL;DR: In this article, a detailed system analysis is performed which allows formulation of simplified hydraulic and cost models covering all significant operational aspects, and the problem is then partitioned into a dynamic source optimisation task and a static network control task.
Abstract: The paper seeks to establish useful and flexible techniques for optimising the operation of water networks and also to extend the range of systems which can be treated. As a preliminary, a detailed systems analysis is performed which allows formulation of simplified hydraulic and cost models covering all significant operational aspects. A problem at present under investigation concerns optimised pump scheduling of single reservoir sources, For this situation practical results are obtained by use of dynamic programming. The method is then extended to cover systems having multiple source supplies external to the network, together with variable closure valves which can be used to control the network operation. Under these conditions it is possible to partition the problem into that of a dynamic source optimisation task and a static network control task. Source optimisation takes account of the relative cost of supply to yield optimal pumping schedules, and network control involves adjustment of the valves to...
TL;DR: In this article, an efficient algorithm for the real-time monthly operation of a multipurpose reservoir is presented, which is a combination of linear programming (used for month-by-month optimization) and dynamic programming(used for annual optimization).
Abstract: An efficient algorithm for the real-time monthly operation of a multipurpose reservoir is presented. The model is a combination of linear programming (used for month-by-month optimization) and dynamic programming (used for annual optimization). The use of parametric linear programming, minimum required beginning-of-month storages, and an iterative solution procedure result in low computer time and computer storage requirements. Low computer storage requirements allow the model to be run on minicomputers. Water and energy maximization, water and energy maximization with flood control considerations, and water and energy maximization for peak demand months are considered. Thus, the model provides the reservoir operator with different choices for annual optimization. The model is applied to Folsom reservoir of the California Central Valley Project and the results are discussed.
TL;DR: An extension of dynamic programming models to include goal objectives is described, and illustrative examples are provided.
Abstract: An extension of dynamic programming models to include goal objectives is described, and illustrative examples are provided.
01 Dec 1984
TL;DR: A general dynamic programming operator model that includes, but is not restricted to, optimization problems is proposed that is inclusive of sequential nonextremization problems.
Abstract: Several authors (Denardo [61 Karp and Held [18], and Bertsekas [3]) have proposed abstract dynamic programming models encompassing a wide variety of sequential optimization problems. The unifying purpose of these models is to impose sufficient conditions on the recursive defimtion of the objective function to guarantee the validity of the solution of the optimization problem by a dynamic programming iteration. In this paper we propose a general dynamic programming operator model that includes, but is not restricted to, optimization problems. Any functional satisfying a certain commutativity condition (which reduces to the principle of optimality in extrermzation problems see Section 2, B2) a-ith the generating operator of the objective recursive function, results in a sequential problem solvable by a dynamic programming iteration. Examples of sequential nonextremization problems fitting t h s framework are the derivation of marginal distributions in arbitrary probability spaces, iterative computation of stageseparated functions defined on general algebra~c systems such as additive commutative semi-groups with distributlve products, generation of symbolic transfer functions, and the Chapman-Kolmogorov equations.
01 Dec 1984
TL;DR: In this paper, a general dynamic programming algorithm for the solution of optimal stochastic control problems concerning a class of discrete event systems is presented, and the algorithm is shown to be optimal in the presence of discrete events.
Abstract: This paper presents a general dynamic programming algorithm for the solution of optimal stochastic control problems concerning a class of discrete event systems.
TL;DR: In this paper, the problem of minimizing the total cost of a pump-pipe system in series is considered and a general mathematical model is formulated and dynamic programming is used to find an optimal solution.
Abstract: In this paper the problem of minimizing the total cost of a pump-pipe system in series is considered. The route of the pipeline and the number of pumping stations are known. The optimization will then consist in determining the control variables, diameter and thickness of the pipe and the size of the pumps. A general mathematical model is formulated and Dynamic Programming is used to find an optimal solution. Practical reasons, derived from the techniques engineers generally use to cope with such problems, and special characteristics of the mathematical structure of the model justified the consideration of particular cases of the system. This analysis, based on Dynamic Programming, enabled us to elaborate a simple heuristic method, condensing those techniques, and supplied sufficient conditions for the heuristic to operate as an optimal procedure. The solution of a realistic example confirms the viability of the conditions developed and tests the formulation (also presented) of the optimization problem by...
01 Jun 1984
TL;DR: This paper is concerned with the optimization of base load production of hydro energy storage plants within a given time interval at a varying tariff rate and two newly developed nonlinear optimization methods were used.
Abstract: This paper is concerned with the optimization of base load production of hydro energy storage plants within a given time interval at a varying tariff rate. For three real world storage plants in Austria, each of them presenting some characteristic difficulty we discuss both mathematical models and numerical techniques. Beside classical techniques as Dynamic Programming and Simulation two newly developed nonlinear optimization methods were used.
TL;DR: In this article, another dynamic programing (DP) based approach is proposed to solve the problem of dimensionality of the objective functions of a four-reservoir operation problem.
Abstract: Reservoir operation problems are complicated by the nonlinearities in the objective functions. The dynamic programing (DP) procedure is often used to solve this problem because of the sequential nature of the decisions involved, but for simultaneous operations of multireservoir systems, other DP-based techniques are frequently found to be more efficient in overcoming the curse of dimensionality problem caused by the interdependencies of the decisions. In this paper, another DP-based procedure is proposed which performs better than the other well-known techniques. The applications of this technique are presented in two different formulations for four-reservoir operation problems. The convergence properties of the algorithm are investigated as revealed from the systematic solutions of a control problem with various dimensions.