scispace - formally typeset
Search or ask a question

Showing papers on "Dynamic programming published in 2005"


Journal ArticleDOI
TL;DR: B bounded sparse dynamic programming (BSDP) is introduced to allow rapid implementation of heuristics approximating to many complex alignment models, and has been incorporated into the freely available sequence alignment program, exonerate.
Abstract: Exhaustive methods of sequence alignment are accurate but slow, whereas heuristic approaches run quickly, but their complexity makes them more difficult to implement. We introduce bounded sparse dynamic programming (BSDP) to allow rapid approximation to exhaustive alignment. This is used within a framework whereby the alignment algorithms are described in terms of their underlying model, to allow automated development of efficient heuristic implementations which may be applied to a general set of sequence comparison problems. The speed and accuracy of this approach compares favourably with existing methods. Examples of its use in the context of genome annotation are given. This system allows rapid implementation of heuristics approximating to many complex alignment models, and has been incorporated into the freely available sequence alignment program, exonerate.

2,292 citations


Journal ArticleDOI
TL;DR: This work considers a robust control problem for a finite-state, finite-action Markov decision process, where uncertainty on the transition matrices is described in terms of possibly nonconvex sets, and shows that perfect duality holds for this problem, and that it can be solved with a variant of the classical dynamic programming algorithm, the "robust dynamic programming" algorithm.
Abstract: Optimal solutions to Markov decision problems may be very sensitive with respect to the state transition probabilities In many practical problems, the estimation of these probabilities is far from accurate Hence, estimation errors are limiting factors in applying Markov decision processes to real-world problems We consider a robust control problem for a finite-state, finite-action Markov decision process, where uncertainty on the transition matrices is described in terms of possibly nonconvex sets We show that perfect duality holds for this problem, and that as a consequence, it can be solved with a variant of the classical dynamic programming algorithm, the "robust dynamic programming" algorithm We show that a particular choice of the uncertainty sets, involving likelihood regions or entropy bounds, leads to both a statistically accurate representation of uncertainty, and a complexity of the robust recursion that is almost the same as that of the classical recursion Hence, robustness can be added at practically no extra computing cost We derive similar results for other uncertainty sets, including one with a finite number of possible values for the transition matrices We describe in a practical path planning example the benefits of using a robust strategy instead of the classical optimal strategy; even if the uncertainty level is only crudely guessed, the robust strategy yields a much better worst-case expected travel time

740 citations


Journal ArticleDOI
TL;DR: A new control strategy called Adaptive Equivalent Consumption Minimization Strategy (A-ECMS) is presented, adding to the ECMS framework an on-the-fly algorithm for the estimation of the equivalence factor according to the driving conditions.

729 citations


Journal ArticleDOI
TL;DR: It is proved that when this set of measures has a certain "rectangularity" property, all of the main results for finite and infinite horizon DP extend to natural robust counterparts.
Abstract: In this paper we propose a robust formulation for discrete time dynamic programming (DP). The objective of the robust formulation is to systematically mitigate the sensitivity of the DP optimal policy to ambiguity in the underlying transition probabilities. The ambiguity is modeled by associating a set of conditional measures with each state-action pair. Consequently, in the robust formulation each policy has a set of measures associated with it. We prove that when this set of measures has a certain "rectangularity" property, all of the main results for finite and infinite horizon DP extend to natural robust counterparts. We discuss techniques from Nilim and El Ghaoui [17] for constructing suitable sets of conditional measures that allow one to efficiently solve for the optimal robust policy. We also show that robust DP is equivalent to stochastic zero-sum games with perfect information.

585 citations


Journal ArticleDOI
TL;DR: An efficient SCUC approach with ac constraints that obtains the minimum system operating cost while maintaining the security of power systems is introduced.
Abstract: In a restructured power market, the independent system operator (ISO) executes the security-constrained unit commitment (SCUC) program to plan a secure and economical hourly generation schedule for the day-ahead market. This paper introduces an efficient SCUC approach with ac constraints that obtains the minimum system operating cost while maintaining the security of power systems. The proposed approach applies the Benders decomposition for separating the unit commitment (UC) in the master problem from the network security check in subproblems. The master problem applies the augmented Lagrangian relaxation (LR) method and dynamic programming (DP) to solve UC. The subproblem checks ac network security constraints for the UC solution to determine whether a converged and secure ac power flow can be obtained. If any network violations arise, corresponding Benders cuts will be formed and added to the master problem for solving the next iteration of UC. The iterative process will continue until ac violations are eliminated and a converged optimal solution is found. In this paper, a six-bus system and the IEEE 118-bus system with 54 units are analyzed to exhibit the effectiveness of the proposed approach.

441 citations


Journal ArticleDOI
TL;DR: The aim of the paper is to give basic theoretical results on the structure of the optimal state-feedback solution and of the value function and to describe how the state- feedback optimal control law can be constructed by combining multiparametric programming and dynamic programming.

372 citations


Proceedings ArticleDOI
12 Dec 2005
TL;DR: DynDE is described, a multipopulation DE algorithm developed specifically to solve dynamic optimization problems that doesn't need any parameter control strategy for the F or CR parameters.
Abstract: This paper presents an approach of using differential evolution (DE) to solve dynamic optimization problems. Careful setting of parameters is necessary for DE algorithms to successfully solve optimization problems. This paper describes DynDE, a multipopulation DE algorithm developed specifically to solve dynamic optimization problems that doesn't need any parameter control strategy for the F or CR parameters. Experimental evidence has been gathered to show that this new algorithm is capable of efficiently solving the moving peaks benchmark.

202 citations


Journal ArticleDOI
TL;DR: In this paper, the authors present an efficient procedure which allows to carry out reliability-based optimization of linear systems subjected to stochastic loading, where the optimization problem is replaced by a sequence of approximate explicit sub-optimization problems that are solved in an efficient manner.

200 citations


Proceedings ArticleDOI
20 Jun 2005
TL;DR: By employing a two-pass dynamic programming technique that performs optimization both along and across the scanlines, the typical inter-scanline inconsistency problem is solved and the stability and efficiency of the optimization are improved significantly.
Abstract: A method for solving dense stereo matching problem is presented in this paper. First, a new generalized ground control points (GGCPs) scheme is introduced, where one or more disparity candidates for the true disparity of each pixel are assigned by local matching using the oriented spatial filters. By allowing "all" pixels to have multiple candidates for their true disparities, GGCPs not only guarantee to provide a sufficient number of starting pixels needed for guiding the subsequent matching process, but also remarkably reduce the risk of false match, improving the previous GCP-based approaches where the number of the selected control points tends to be inversely proportional to the reliability. Second, by employing a two-pass dynamic programming technique that performs optimization both along and across the scanlines, we solve the typical inter-scanline inconsistency problem. Moreover, combined with the GGCPs, the stability and efficiency of the optimization are improved significantly. Experimental results for the standard data sets show that the proposed algorithm achieves comparable results to the state-of-the-arts with much less computational cost.

194 citations


Journal ArticleDOI
TL;DR: This research proposes a procedure for identifying dynamic routing policies in stochastic transportation networks using the Laplace transform and its numerical inversion to reduce the computational cost of evaluating the convolution integrals that result from the successive approximation procedure.
Abstract: This research proposes a procedure for identifying dynamic routing policies in stochastic transportation networks. It addresses the problem of maximizing the probability of arriving on time. Given a current location (node), the goal is to identify the next node to visit so that the probability of arriving at the destination by time t or sooner is maximized, given the probability density functions for the link travel times. The Bellman principle of optimality is applied to formulate the mathematical model of this problem. The unknown functions describing the maximum probability of arriving on time are estimated accurately for a few sample networks by using the Picard method of successive approximations. The maximum probabilities can be evaluated without enumerating the network paths. The Laplace transform and its numerical inversion are introduced to reduce the computational cost of evaluating the convolution integrals that result from the successive approximation procedure.

183 citations


Journal ArticleDOI
TL;DR: This paper proposes local search algorithms for the vehicle routing problem with soft time-window constraints and shows that this problem can be efficiently solved by using dynamic programming, which is then incorporated in the algorithm.
Abstract: We propose local search algorithms for the vehicle routing problem with soft time-window constraints. The time-window constraint for each customer is treated as a penalty function, which is very general in the sense that it can be nonconvex and discontinuous as long as it is piecewise linear. In our algorithm, we use local search to assign customers to vehicles and to find orders of customers for vehicles to visit. Our algorithm employs an advanced neighborhood, called the cyclic-exchange neighborhood, in addition to standard neighborhoods for the vehicle routing problem. After fixing the order of customers for a vehicle to visit, we must determine the optimal start times of processing at customers so that the total penalty is minimized. We show that this problem can be efficiently solved by using dynamic programming, which is then incorporated in our algorithm. We report computational results for various benchmark instances of the vehicle routing problem. The generality of time-window constraints allows us to handle a wide variety of scheduling problems. As an example, we mention in this paper an application to a production scheduling problem with inventory cost, and report computational results for real-world instances.

Book
10 Jan 2005
TL;DR: In this paper, the authors define necessary and sufficient conditions for a general class of control problems, including linear optimal control problems and isoperimetric control problems with one state variable and one control variable.
Abstract: 1. Essential elements of continuous time dynamic optimization 2. Necessary conditions for a simplified control problem 3. Concavity and sufficiency in optimal control problems 4. The maximum principle and economic interpretations 5. Linear optimal control problems 6. Necessary and sufficient conditions for a general class of control problems 7. Necessary and sufficient conditions for isoperimetric problems 8. Economic characterization of reciprocal isoperimetric problems 9. The dynamic envelope theorem and economic interpretations 10. The dynamic envelope theorem and transversality conditions 11. Comparative dynamics via envelope methods 12. Discounting, current values, and time consistency 13. Local stability and phase portraits of autonomous differential equations 14. Necessary and sufficient conditions for infinite horizon control problems 15. The neoclassical optimal economic growth model 16. A dynamic limit pricing model of the firm 17. The adjustment cost model of the firm 18. Qualitative properties of infinite horizon optimal control problems with one state variable and one control variable 19. Dynamic programming and the Hamilton-Jacobi-Bellman equation 20. Intertemporal duality in the adjustment cost model of the firm.

Journal ArticleDOI
TL;DR: A stochastic gradient algorithm and approximate dynamic programming ideas are combined to improve the initial booking limits of nested booking limits and suggest that the proposed algorithm can lead to practically significant revenue enhancements.
Abstract: Deterministic mathematical programming models that capture network effects play a predominant role in the theory and practice of airline revenue management. These models do not address important issues like demand uncertainty, nesting, and the dynamic nature of the booking process. Alternatively, the network problem can be broken down into leg-based problems for which there are satisfactory solution methods, but this approach cannot be expected to capture all relevant network aspects. In this paper, we propose a new algorithm that addresses these issues. Starting with any nested booking-limit policy, we combine a stochastic gradient algorithm and approximate dynamic programming ideas to improve the initial booking limits. Preliminary simulation experiments suggest that the proposed algorithm can lead to practically significant revenue enhancements.

Posted Content
TL;DR: An optimal ex post probability density for wealth in two leading cases (log and linear utility) is derived and a general approach for handling other cases numerically is laid out and displays and discusses numerical solutions for other utility functions.
Abstract: The literature applying information-theoretic ideas to economics has so far considered only Gaussian uncertainty. Ex post Gaussian uncertainty can be justified as optimal when the associated optimization problem is linear-quadratic, but the literature has often assumed Gaussian uncertainty even where it cannot be justified as optimal. This paper considers a simple two-period optimal saving problem with a Shannon capacity constraint and non-quadratic utility. It derives an optimal ex post probability density for wealth in two leading cases (log and linear utility) and lays out a general approach for handling other cases numerically. It displays and discusses numerical solutions for other utility functions, and considers the feasibility of extending this paper?s approaches to general non-LQ dynamic programming problems. The introduction of the paper discusses approaches that have been taken in the existing literature to applying Shannon capacity to economic modeling, making criticisms and suggesting promising directions for further progress.

Journal ArticleDOI
TL;DR: An adaptive sampling algorithm that adaptively chooses which action to sample as the sampling process proceeds and generates an asymptotically unbiased estimator, whose bias is bounded by a quantity that converges to zero at rate (lnN)/ N.
Abstract: Based on recent results for multiarmed bandit problems, we propose an adaptive sampling algorithm that approximates the optimal value of a finite-horizon Markov decision process (MDP) with finite state and action spaces. The algorithm adaptively chooses which action to sample as the sampling process proceeds and generates an asymptotically unbiased estimator, whose bias is bounded by a quantity that converges to zero at rate (lnN)/ N, whereN is the total number of samples that are used per state sampled in each stage. The worst-case running-time complexity of the algorithm isO(( |A|N) H ), independent of the size of the state space, where | A| is the size of the action space andH is the horizon length. The algorithm can be used to create an approximate receding horizon control to solve infinite-horizon MDPs. To illustrate the algorithm, computational results are reported on simple examples from inventory control.

Journal ArticleDOI
TL;DR: An attempt is made to review the literature on optimizing machining parameters in turning processes and the latest techniques for optimization include fuzzy logic, scatter search technique, genetic algorithm, Taguchi technique and response surface methodology.
Abstract: In this paper an attempt is made to review the literature on optimizing machining parameters in turning processes. Various conventional techniques employed for machining optimization include geometric programming, geometric plus linear programming, goal programming, sequential unconstrained minimization technique, dynamic programming etc. The latest techniques for optimization include fuzzy logic, scatter search technique, genetic algorithm, Taguchi technique and response surface methodology.

Journal ArticleDOI
TL;DR: This paper presents an algorithm combining dynamic programming and genetic search for solving a dynamic facility layout problem, which may change from one period in time to the next.

Journal ArticleDOI
TL;DR: A two-phase exact algorithm based on dynamic programming (DP) is proposed that finds the best routes for a fleet of trucks that is suitable for solving large size problems.
Abstract: Container movement by trucks with time constraints at origins and destinations is modeled as an asymmetric “multi-Traveling Salesmen Problem with Time Windows” (m-TSPTW) with social constraints. A two-phase exact algorithm based on dynamic programming (DP) is proposed that finds the best routes for a fleet of trucks. Since the m-TSPTW problem is NP-hard, the computational time for optimally solving large size problems becomes prohibitive. For large size problems, we develop a hybrid methodology consisting of DP in conjunction with genetic algorithms. The developed algorithms are compared with an insertion heuristic method. Computational results demonstrate the efficiency of the developed algorithms.

Journal ArticleDOI
TL;DR: A fast heuristic model based on dynamic programming is proposed for the search of FIS shape, which searches the optimal focus measure in the whole image volume, instead of the small volume as adopted in previous methods.
Abstract: The most popular shape from focus (SFF) methods in the literature are based on the concept of focused image surface (FIS)-the surface formed by the best focus points. According to paraxial-geometric optics, there is one-to-one correspondence between the shape of an object and the shape of its FIS. Therefore, the problem of three-dimensional (3-D) shape recovery from image focus can be described as the problem of determining the shape of the FIS. The conventional SFF method is inaccurate because of piecewise constant approximation of the FIS. The SFF method based on the FIS has shown better results by exhaustive search of the FIS shape using planar surface approximation at the cost of considerably higher computations. In this paper, search of the FIS shape is presented as an optimization problem, i.e., maximization of the focus measure in the 3-D image volume. The proposed method searches the optimal focus measure in the whole image volume, instead of the small volume as adopted in previous methods. The dynamic programming, instead of the approximation techniques, is used to search the optimal FIS shape. A direct application of dynamic programming on a 3-D data is impractical, because of higher computational complexity. Therefore a fast heuristic model based on dynamic programming is proposed for the search of FIS shape. The shape recovery results of the new method are better than previous methods. The proposed algorithm is significantly faster than the FIS algorithm, but a little slower than the conventional algorithm.

Journal ArticleDOI
TL;DR: It is proved that an $(s, S)$ policy is optimal in a continuous-review stochastic inventory model with a fixed ordering cost when the demand is a mixture of a diffusion process and a compound Poisson process with exponentially distributed jump sizes.
Abstract: We prove that an $(s, S)$ policy is optimal in a continuous-review stochastic inventory model with a fixed ordering cost when the demand is a mixture of (i) a diffusion process and a compound Poisson process with exponentially distributed jump sizes, and (ii) a constant demand and a compound Poisson process. The proof uses the theory of impulse control. The Bellman equation of dynamic programming for such a problem reduces to a set of quasi-variational inequalities (QVI). An analytical study of the QVI leads to showing the existence of an optimal policy as well as the optimality of an $(s, S)$ policy. Finally, the combination of a diffusion and a general compound Poisson demand is not completely solved. We explain the difficulties and what remains open. We also provide a numerical example for the general case.

Journal ArticleDOI
TL;DR: In this article, the authors studied the problem of a company that adjusts its stochastic production capacity in reversible investments with controls of expansion and contraction, and provided a complete solution with explicit expressions of the value functions and the optimal controls: the company activates its production once a fixed entry threshold of the capacity is reached, and invests in capital so as to maintain its capacity in a closed bounded interval.

Journal ArticleDOI
TL;DR: It is shown that the sub-problems of the reliability & redundancy allocation problem are equivalent to one-dimensional knapsack problems which can be solved in pseudopolynomial time with a dynamic programming approach, and that the obtained method YCC converges toward an optimal solution.
Abstract: Reliability & redundancy allocation is one of the most frequently encountered problems in system design. This problem is subject to constraints related to the design, such as required structural, physical, and technical characteristics; and the components available in the market. This last constraint implies that system components, and their reliability, must belong to a finite set. For a parallel-series system, we show that the problem can be modeled as an integer linear program, and solved by a decomposition approach. The problem is decomposed into as many sub-problems as subsystems, one sub-problem for each subsystem. The sub-problem for a given subsystem consists of determining the number of components of each type in order to reach a given reliability target with a minimum cost. The global problem consists of determining the reliability target of subsystems. We show that the sub-problems are equivalent to one-dimensional knapsack problems which can be solved in pseudopolynomial time with a dynamic programming approach. We show that the global problem can also be solved by a dynamic programming technique. We also show that the obtained method YCC converges toward an optimal solution.

Journal ArticleDOI
TL;DR: In this article, an optimization algorithm should reflect the problem's dynamics and explicitly take into account that changes to the current solution are to be expected, i.e. easily adjustable if necessary in the case of problem changes.
Abstract: Many real-world optimization problems change over time and require frequent re-optimization. We suggest that in such environments, an optimization algorithm should reflect the problem's dynamics and explicitly take into account that changes to the current solution are to be expected. We claim that this can be achieved by having the optimization algorithm search for solutions that are not only good, but also flexible, i.e. easily adjustable if necessary in the case of problem changes. For the example of a job-shop with jobs arriving non-deterministically over time, we demonstrate that avoiding early idle times increases flexibility, and thus that the incorporation of an early idle time penalty as secondary objective into the scheduling algorithm can greatly enhance the overall system performance.

Journal ArticleDOI
TL;DR: In this paper, an algorithm for dynamic response optimization transforming dynamic loads into equivalent static loads has been proposed, which is applied to the optimization of flexible multibody dynamic systems, where the equivalent static load is derived from the equations of motion for a flexible multi-body dynamic system.
Abstract: Recently, an algorithm for dynamic response optimization transforming dynamic loads into equivalent static loads has been proposed. In later research, it was proved that the solution obtained by the algorithm satisfies the Karush-Kuhn-Tucker necessary conditions. In the present research, the proposed algorithm is applied to the optimization of flexible multibody dynamic systems. The equivalent static load is derived from the equations of motion for a flexible multibody dynamic system. The equivalent load is utilized in sequential static response optimization of the flexible mutibody dynamic system. In the end, the converged solution of the sequential static response optimization is the solution of the original dynamic response optimization. Some standard examples are solved to show the feasibility and efficiency of the proposed method. The control arm of an automobile suspension system is optimized as a practical problem. The results are discussed regarding the application of the proposed algorithm to flexible multibody dynamic systems.

Journal ArticleDOI
TL;DR: This paper seeks to enhance the real options methodology developed by Copeland and Antikarov with traditional decision analysis tools to propose a discrete time method that allows the problem to be specified and solved with off the shelf decision analysis software.
Abstract: In this paper we seek to enhance the real options methodology developed by Copeland and Antikarov (2001) with traditional decision analysis tools to propose a discrete time method that allows the problem to be specified and solved with off the shelf decision analysis software. This method uses dynamic programming with an innovative algorithm to model the project’s stochastic process and real options with decision trees. The method is computationally intense, but simpler and more intuitive than traditional methods, thus allowing for greater flexibility in the modeling of the problem.

Book ChapterDOI
19 Jun 2005
TL;DR: This paper presents another algorithm, DNAPack, based on dynamic programming, which compresses DNA slightly better, while the cost of dynamic programming is almost negligible.
Abstract: Standard compression algorithms are not able to compress DNA sequences. Recently, new algorithms have been introduced specifically for this purpose, often using detection of long approximate repeats. In this paper, we present another algorithm, DNAPack, based on dynamic programming. In comparison with former existing programs, it compresses DNA slightly better, while the cost of dynamic programming is almost negligible.

Journal ArticleDOI
TL;DR: In this article, a survey of stochastic control with respect to diffusion processes is presented, with a view towards applications and numerical issues and open questions, including degenerate singular control problems.
Abstract: This paper is a survey on some recent aspects and developments in stochastic control. We discuss the two main historical approaches, Bellman's optimality principle and Pontryagin's maximum principle, and their modern exposition with viscosity solutions and backward stochastic differential equations. Some original proofs are presented in a unifying context including degenerate singular control problems. We emphasize key results on characterization of optimal control for diffusion processes, with a view towards applications. Some examples in finance are detailed with their explicit solutions. We also discuss numerical issues and open questions.

BookDOI
TL;DR: An algorithm to compute the optimal solution for the finite time case where the algorithm combines a dynamic programming exploration strategy with multi-parametric linear programming and basic polyhedral manipulation and the equivalence of the dynamic programming generated solution with the solution to the infinite time optimal control problem is shown.
Abstract: We consider the constrained finite and infinite time optimal control problem for the class of discrete-time linear piecewise affine systems. When a linear performance index is used the finite and infinite time optimal solution is a piecewise affine state feedback control law. In this paper we present an algorithm to compute the optimal solution for the finite time case where the algorithm combines a dynamic programming exploration strategy with multi-parametric linear programming and basic polyhedral manipulation. We extend the ideas to the infinite time case and show the equivalence of the dynamic programming generated solution with the solution to the infinite time optimal control problem.

Journal ArticleDOI
TL;DR: The proposed problem is equivalently transformed into a static multiobjective shortest path problem in an acyclic network reconstructed by the space-time network technique and an efficient dynamic programming method is developed.
Abstract: This paper proposes a novel vehicle routing and scheduling problem in transporting hazardous materials for networks with multiple time-varying attributes. It actually aims to identify all nondominated time-varying paths with departure times at the origin and waiting times at intermediate nodes along these paths for a given pair of origin and destination, subject to three kinds of practical constraints: Limited operational time period, and service, and waiting time window constraints at a node. Based on the assumption of linear waiting attributes at a node, the proposed problem can be equivalently transformed into a static multiobjective shortest path problem in an acyclic network reconstructed by the space-time network technique. An efficient dynamic programming method is then developed. In addition, two numerical examples and a case study are carried out to demonstrate the methodology proposed in this paper.

Journal ArticleDOI
TL;DR: A scheduling model for optimal production sequencing in a flexible assembly system that is modeled using timed Petri nets and task scheduling is solved with a dynamic programming algorithm to minimize the completion time for a single product or a batch of products.
Abstract: This paper investigates a scheduling model for optimal production sequencing in a flexible assembly system. The system features a set of machines working together in the same workspace, with each machine performing a subset of operations. Three constraints are considered: (1) the precedence relation among the operations specified by the assembly tree; (2) working space that limits concurrent operations; and (3) the variation of process time. The objective is to find both a feasible assignment of operations to machines and schedule tasks in order to minimize the completion time for a single product or a batch of products. The assembly process is modeled using timed Petri nets and task scheduling is solved with a dynamic programming algorithm. The method calculates the time required precisely. A detailed case study is discussed to show the effectiveness of the model and algorithm.