scispace - formally typeset
Search or ask a question

Showing papers on "Stochastic programming published in 1989"


Journal Article

1,389 citations


Book ChapterDOI
01 Jan 1989
TL;DR: In the long history of mathematics, stochastic optimal control is a rather recent development using Bellman's Principle of Optimality along with measure-theoretic and functional-analytic methods.
Abstract: In the long history of mathematics, stochastic optimal control is a rather recent development. Using Bellman’s Principle of Optimality along with measure-theoretic and functional-analytic methods, several mathematicians such as H. Kushner, W. Fleming, R. Rishel. W.M. Wonham and J.M. Bismut, among many others, made important contributions to this new area of mathematical research during the 1960s and early 1970s. For a complete mathematical exposition of the continuous time case see Fleming and Rishel (1975) and for the discrete time case see Bertsekas and Shreve (1978).

415 citations


01 Sep 1989
TL;DR: It is shown how a TD METHOD can beunderstood as a NOVEL SYNTHESIS of CONCEPTS from the theORY of STOCHASTIC DYNAMIC PROGRAMMING, which is the standard method for solving decision-making problems in binary systems.
Abstract: IN THIS REPORT WE SHOW HOW THE CLASS OF ADAPTIVE PREDICTION METHODS THAT SUTTON CALLED "TEMPORAL DIFFERENCE", OR TD, METHODS ARE RELATED TO THE THE- ORY OF SEQUENTIAL DECISION MAKING. TD METHODS HAVE BEEN USED AS "ADAPTIVE CRITICS" IN CONNECTIONIST LEARNING SYSTEMS,AND HAVE BEEN PROPOSED AS MODELS OF ANIMAL LEARNING IN CLASSICAL CONDITIONING EXPERIMENTS. HERE WE RELATE TD METHODS TO DECISION TASKS FORMULATED IN TERMS OF A STOCHASTIC DYNAMICAL SYSTEM WHOSE BEHAVIOR UNFOLDS OVER TIME UNDER THE INFLUENCE OF A DECISION MAKER''S ACTIONS. STRATEGIES ARE SOUGHT FOR SELECTING ACTIONS SO AS TO MAXI- MIZE A MEASURE OF LONG-TERM PAYOFF GAIN. MATHEMATICALLY, TASKS SUCH AS THIS CAN BE FORMULATED AS MARKOVIAN DECISION PROBLEMS, AND NUMEROUS METHODS HAVE BEEN PROPOSED FOR LEARNING HOW TO SOLVE SUCH PROBLEMS. WE SHOW HOW A TD METHOD CAN BE UNDERSTOOD AS A NOVEL SYNTHESIS OF CONCEPTS FROM THE THEORY OF STOCHASTIC DYNAMIC PROGRAMMING, WHICH COMPRISES THE STANDARD METHOD FOR SOLVING SUCH TASKS WHEN A MODEL OF THE DYNAMICAL SYSTEM IS AVAILABLE, AND THE THEORY OF PARAMETER ESTIMATION, WHICH PROVIDES THE APPROPRIATE CONTEXT FOR STUDYING LEARNING RULES IN THE FORM OF EQUATIONS FOR UPDATING ASSOCIA- TIVE STRENGTHS IN BEHAVIORAL MODELS, OR CONNECTION WEIGHTS IN CONNECTIONIST NETWORKS. BECAUSE THIS REPORT IS ORIENTED PRIMARILY TOWARD THE NON-ENGINEER INTERESTED IN ANIMAL LEARNING, IT PRESENTS TUTORIALS ON STOCHASTIC SEQUEN- TIAL DECISION TASKS, STOCHASTIC DYNAMIC PROGRAMMING, AND PARAMETER ESTIMATI

342 citations


Journal ArticleDOI
TL;DR: This paper considers the vehicle routing problem with stochastic demands, and a new solution framework for the problem using Markovian decision processes is presented.
Abstract: This paper considers the vehicle routing problem with stochastic demands. The objective is to provide an overview of this problem, and to examine a variety of solution methodologies. The concepts and the main issues are reviewed along with some properties of optimal solutions. The existing stochastic mathematical programming formulations are presented and compared and a new formulation is proposed. A new solution framework for the problem using Markovian decision processes is then presented.

290 citations


Journal ArticleDOI
TL;DR: An algorithm for calculating optimal operating strategies in a multi-reservoir hydroelectric system, which can take into account inflow stochasticity and does not require discretization of the state space is described.

228 citations


Posted Content
TL;DR: A "nested fixed point" algorithm is applied that converts the dynamic programming problem into the problem of repeatedly recomputing the fixed point to a contraction mapping operator as a subroutine of a standard nonlinear maximum likelihood program.
Abstract: This paper formulates a model of retirement behavior based on the solution to a stochastic dynamic programming problem. The workers objective is to maximize expected discounted utility over his remaining lifetime. At each time period the worker chooses how much to consume and whether to work full-time, part-time, or exit the labor force. The model accounts for the sequential nature f the retirement decision problem, and the role of expectations of uncertain future variables such as the worker's future lifespan, health status, marital and family status, employment status, as well as earnings from employment, assets, and social security retirement, disability and medicare payments. This paper applies a "nested fixed point" algorithm that converts the dynamic programming problem into the problem of repeatedly recomputing the fixed point to a contraction mapping operator as a subroutine of a standard nonlinear maximum likelihood program. The goal of the paper is to demonstrate that a fairly complex and realistic formulation of the retirement problem can be estimated using this algorithm and a current generation supercomputer, the Cray-2.

163 citations


Journal ArticleDOI
TL;DR: In this paper, the authors investigate the asymptotic behavior of estimators of the optimal value and optimal solutions of a stochastic program and show that in the presence of inequality constraints, the estimators are not normal in general.
Abstract: The aim of this article is to investigate the asymptotic behaviour of estimators of the optimal value and optimal solutions of a stochastic program. These estimators are closely related to the $M$-estimators introduced by Huber (1964). The parameter set of feasible solutions is supposed to be defined by a number of equality and inequality constraints. It will be shown that in the presence of inequality constraints the estimators are not asymptotically normal in general. Maximum likelihood and robust regression methods will be discussed as examples.

131 citations


Journal ArticleDOI
TL;DR: In this article, the authors describe and compare stochastic network optimization models for investment planning under uncertainty, focusing on multi-period a sset allocation and active portfolio management problems.
Abstract: We describe and compare stochastic network optimization models for investment planning under uncertainty. Emphasis is placed on multiperiod a sset allocation and active portfolio management problems. Myopic as well as multiple period models are considered. In the case of multiperiod models, the uncertainty in asset returns filters into the constraint coefficient matrix, yielding a multi-scenario program formulation. Different scenario generation procedures are examined. The use of utility functions to reflect risk bearing attitudes results in nonlinear stochastic network models. We adopt a newly proposed decomposition procedure for solving these multiperiod stochastic programs. The performance of the models in simulations based on historical data is discussed.

123 citations


Journal ArticleDOI
TL;DR: In this paper, a unified approach to various problems of structural optimization is presented, based on a combination of mathematical models of different complexity, which describe the behaviour of a designed structure and are connected with the sequential approximation of design problem constraints and/or an objective function.
Abstract: A unified approach to various problems of structural optimization is presented. It is based on a combination of mathematical models of different complexity. The models describe the behaviour of a designed structure. From the computational point of view, it is connected with the sequential approximation of design problem constraints and/or an objective function. In each step, a subregion of the initial search region in the space of design variables is chosen. In this subregion, various points (designs) are selected, for which response analyses are carried out using a numerical method (mostly FEM). Using the least-squares method, analytical expressions are formulated, which then replace the initial problem functions. They are used as functions of a particular mathematical programming problem. The size and location of sequential subregions may be changed according to the result of the search. The choice of one particular form of the analytical expressions is described. The application of the approach is shown by means of test examples and comparison with other optimization techniques is presented.

122 citations


Book ChapterDOI
01 Apr 1989
TL;DR: Most systems that need to be controlled or analyzed involve some level of uncertainty about the value to assign to some of the parameters, if not about the actual layout of certain subcomponents of the system.
Abstract: Most systems that need to be controlled or analyzed involve some level of uncertainty about the value to assign to some of the parameters, if not about the actual layout of certain subcomponents of the system In many situations not much is lost by assigning “reasonable” values to these parameters or by choosing a particular design In other instances ignoring uncertainty may very well lead to totally misleading solutions that would invalidate any of the implications one may wish to draw from the analysis

97 citations


Journal ArticleDOI
TL;DR: In this article, a dual of this problem is formulated to obtain an implementable procedure to calculate the bound, which can often be used when other suggested upper bounds are intractable.
Abstract: Separable sublinear functions are used to provide upper bounds on the recourse function of a stochastic program. The resulting problem's objective involves the inf-convolution of convex functions. A dual of this problem is formulated to obtain an implementable procedure to calculate the bound. Function evaluations for the resulting convex program only require a small number of single integrations in contrast with previous upper bounds that require a number of function evaluations that grows exponentially in the number of random variables. The sublinear bound can often be used when other suggested upper bounds are intractable. Computational results indicate that the sublinear approximation provides good, efficient bounds on the stochastic program objective value.

Journal ArticleDOI
TL;DR: Application of a sequential quadratic programming method for dynamic response optimization and optimal control problems and several methods to treat time-dependent constraints are described.
Abstract: This paper describes application of a sequential quadratic programming method for dynamic response optimization and optimal control problems. Several methods to treat time-dependent constraints are described

Book ChapterDOI
01 Jan 1989
TL;DR: The problem of designing price-guided resource allocation processes to cope with increasing returns has been a topic of continuing interest as discussed by the authors, and it is worth noting that neither John Keynes, the editor of the Economic Journal who was most appreciative of Ramsey's talents, nor the subsequent writers on "growth theory" in Cambridge, England (nor, for that matter, those in Cambridge Massachusetts), have made any precise suggestion towards incorporating increasing returns in a Ramsey-type exercise.
Abstract: An editorial note in the Economic Journal (May 1930) reported the death of Frank Ramsey, and his 1928 paper was described as ‘one of the most remarkable contributions to mathematical economics ever made’. In the same issue the editor organized a symposium on increasing returns and the representative firm. This symposium seems to be a natural follow-up of a number of papers published by the Journal during 1926–8, including the well-known article of Allyn Young (1928) that is still available, and duly remembered. The problems of equilibrium of a firm under increasing returns, or more generally, of designing price-guided resource allocation processes to cope with increasing returns, has since been a topic of continuing interest. Ramsey’s contribution was enshrined as a durable piece with a resurgence of interest in intertemporal economics in the fifties. But neither John Keynes, the editor of the Economic Journal who was most appreciative of Ramsey’s talents, neither the subsequent writers on ‘growth theory’ in Cambridge, England (nor, for that matter, those in Cambridge, Massachusetts), have made any precise suggestion towards incorporating increasing returns in a Ramsey-type exercise.


Book ChapterDOI
01 Jan 1989
TL;DR: The stochastic programming model can be viewed as an extension of the linear and nonlinear programming models to decision models where the coefficients that are not known with certainty have been given a probabilistic representation as discussed by the authors.
Abstract: Publisher Summary This chapter focuses on stochastic programming. The stochastic programming model can be viewed as an extension of the linear and nonlinear programming models to decision models where the coefficients that are not known with certainty have been given a probabilistic representation. In the context of the mathematical programming models, some versions of this model were introduced and there had been a number of simple stochastic programming models, which had been formulated in inventory theory—micro-economics and system maintenance. The chapter reviews the useful properties of expectation functionals and analyzes the type of constraints and objective functions that arise in various stochastic programming. The questions of sensitivity and stability of the solution with respect to perturbations of the underlying probability measure, and the implications of these results for stochastic programs are presented in the chapter.

Journal ArticleDOI
TL;DR: For stochastic dynamic programming models with compact, history-dependent action sets, sufficient conditions are given for the existence of a topology such that the space of policies is compact, and the expected reward functionals are upper semicontinuous as mentioned in this paper.

Journal ArticleDOI
TL;DR: In this paper, the authors provide a method that can be used to estimate the volume of water associated with a given reliability for each use of water when the proportion of releases allocated to each use is known.
Abstract: Releases from a reservoir may be allocated to a number of uses, each of which may require a given volume of water at a different reliability. The paper provides a method that can be used to estimate the volume of water associated with a given reliability for each use of water when the proportion of releases allocated to each use is known. These results can be used to evaluate the meeting of specified objectives under a published release policy derived by stationary stochastic dynamic programming. The results can also be used to solve water allocation problems when the probability distribution of available water is known (or can be estimated) and water has multiple uses, each of which has different volume and reliability requirements.

Journal ArticleDOI
TL;DR: In this article, the authors address a so-called assemble-to-order environment, where components must, due to the leadtimes, be ordered prior to knowing the end item demand levels (assumed poisson distributed), but final assemblies can be completed after knowing the demands.

Book
01 Jan 1989
TL;DR: Time Series and Econometric Models: Examples.
Abstract: Deterministic Models and Their Control Problems. Stochastic Models. Stochastic Control Problems. Time Series and Econometric Models: Examples. Estimation. Convergence Questions. Adaptive Control Systems and Bayesian Optimal Control Problems. Linear Rational Expectations Models. Approximations in Sequential Decision Processes. References. Appendix: Markov Processes.

Book ChapterDOI
01 Jan 1989
TL;DR: This chapter presents the mathematical foundations of integer optimization models and the algorithms that can be used to solve them, and discusses the formulations that are good with respect to the efficiency of solving them.
Abstract: Publisher Summary This chapter presents the mathematical foundations of integer optimization models and the algorithms that can be used to solve them. Integer programming deals with problems of maximizing or minimizing a function of many variables subject to inequality and equality constraints, and integrality restrictions on some or all of the variables. There are applications in mathematics to the subjects of combinatorics, graph theory, and logic. Statistical applications include problems of data analysis and reliability. Scientific applications involve problems in molecular biology, high energy physics, and X-ray crystallography. A political application concerns the division of a region into election districts. Some of these discrete optimization models are described in the chapter. The chapter focuses on problems using integer programming models, and discusses the formulations that are good with respect to the efficiency of solving them. Some important theoretical aspects of integer programming models, including fundamental relationships between integer and linear programs, and computational complexity are also discussed in this chapter.

Journal ArticleDOI
TL;DR: In this paper, a decomposition-coordination methodology for solving the long-term optimal scheduling of interconnected multireservoir power systems with stochastic river inflows is presented.
Abstract: A novel decomposition-coordination methodology for solving the long-term optimal scheduling of interconnected multireservoir power systems with stochastic river inflows is presented. The problem (with M reservoirs and N regional power systems) is composed of M hydro and N thermal subproblems. The optimization of the individual hydro subproblem can be solved by two-state stochastic dynamic programming taking into account the stochasticity and time series correlation of the river inflows. The thermal and hydro coordination is accomplished by successively updating dual variables so that the total average hydro and thermal generations meet the predicted system load demands. A computer program has been developed and tested for the Central China and East China Interconnected Power System with up to nine reservoirs. Numerical results show the effectiveness of the proposed method as compared to existing methods. >

Journal ArticleDOI
TL;DR: The linear goal programming method with the steps required to obtain the goal programming solution are discussed and the linearization of the general nonlinear optimization problem along with the flow chart of the algorithm are presented.

Book ChapterDOI
01 Jan 1989
TL;DR: This chapter focuses on global optimization, and the variety of techniques proposed is impressive, but their relative merits have neither been analyzed in a systematic manner nor properly investigated in computational experiments.
Abstract: Publisher Summary This chapter focuses on global optimization. The problem of designing algorithms that distinguish between the local optima and locate the best possible one is known as the “global optimization problem.” Any method for global optimization has to account for the fact that a numerical procedure can never produce more than approximate answers. Irrespective of whether a global optimization method is deterministic or stochastic, it always aims for an appropriate convergence guarantee. A natural approach to solve the global optimization problem is through an appropriate generalization of branch and bound methods. A deterministic approach can be shown to be optimal in the worst case sense, whereas a stochastic approach can be shown to be optimal in the expected utility sense, but neither method can be recommended unless evaluations of the original function f are very expensive. Global optimization as a research area is still on its way to maturity. The variety of techniques proposed is impressive, but their relative merits have neither been analyzed in a systematic manner nor properly investigated in computational experiments.

Journal ArticleDOI
TL;DR: A number of procedures to reduce the size of the recourse problem are described, including a procedure for generating efficiently the feasibility cuts, and it is shown that further reductions are possible if more information about the node-types is taken into account.
Abstract: Preprocessing can speed up the solution procedures for two-stage stochastic programming. We consider the case when the second-stage problem is a pure, uncapacitated network. We describe a number of procedures to reduce the size of the recourse problem. We describe a procedure for generating efficiently the (induced) feasibility cuts, and show that further reductions are possible if more information about the node-types is taken into account. We also investigate network collapsing techniques that would simplify the work required to find both optimality cuts and feasibility cuts, if we had not yet reduced the problem to one with relatively complete recourse. Computational results confirm that substantial savings are possible. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

Proceedings ArticleDOI
Byron Dom1, W. Niblack1, J. Sheinvald1
04 Jun 1989
TL;DR: Two forms of stochastic complexity, minimum description length and an integral form, are applied to the problem of feature selection and give superior results, as measured by their ability to select subsets of relevant features, as well as probability of error computed based on the selected feature subset.
Abstract: The application of J. Rissanen's theory (1986) of stochastic complexity to the problem of features selection in statistical pattern recognition (SPR) is discussed. Stochastic complexity provides a general framework for statistical problems such as coding, prediction, estimation, and classification. A brief review of the SPR paradigm and traditional methods of feature selection is presented, followed by a discussion of the basic of stochastic complexity. Two forms of stochastic complexity, minimum description length and an integral form, are applied to the problem of feature selection. Experimental results using simulated data generated with Gaussian distributions are given and compared with results from cross validation, a traditional technique. The stochastic complexity measures give superior results, as measured by their ability to select subsets of relevant features, as well as probability of error computed based on the selected feature subset. >

Journal ArticleDOI
S. Chowdhury1
TL;DR: A class of combinatorial optimization problems dealing with placing circuit elements on a single row in order to minimize certain costs associated with the placements is formulated and solved.
Abstract: A class of combinatorial optimization problems dealing with placing circuit elements on a single row in order to minimize certain costs associated with the placements is formulated and solved. This analytical formulation of the linear placement problem proceeds with the restriction that all nets are two-point nets. The objective function considered is the sum of the squared wire-lengths. The properties of the formulation are discussed and used to show why certain mathematical programming techniques fail in solving the problems. Solution techniques are presented wherein the search for an optimal solution proceeds within the infeasible region and moves toward the feasible region following the trajectories in which the cost (objective function) tends to be optimal. This important difference of the technique from the previously known heuristics and the associated analysis of complex mathematical structures of the linear placement problems is felt to be important in probing further research in combinatorial optimization problems. >

Proceedings ArticleDOI
Matsuba1
01 Jan 1989
TL;DR: A simulated annealing method based on stochastic dynamic programming is proposed for obtaining a rapid convergence to a global minimum of multivariable optimization problems.
Abstract: A simulated annealing method based on stochastic dynamic programming is proposed for obtaining a rapid convergence to a global minimum of multivariable optimization problems. A central concern is to provide appropriate temperature values that determine the convergence of the cooling schedule. Using the stochastic dynamic programming method, the cooling schedule is derived by minimizing the time that the system requires to reach the global minimum from an initial state, Monte Carlo simulation shows that the present schedule gives good near-optimal solutions. >

Book ChapterDOI
01 Jan 1989
TL;DR: This work has shown that in selected applications, the assumption that diffusion processes are described by diffusion processes, whose continuous sample paths can be represented by Ito integrals, can be relaxed to include both non-Markov path-dependent processes and Poisson-directed jump processes.
Abstract: Models in which agents can revise their decisions continuously in time have proved fruitful in the analysis of economic problems involving intertemporal choice under uncertainty (cf. Malliaris and Brock, 1982). These models frequently produce significantly sharper results than can be derived from their discrete-time counterparts. In the majority of such cases, the dynamics of the underlying system are described by diffusion processes, whose continuous sample paths can be represented by Ito integrals. However, in selected applications, this assumption can be relaxed to include both non-Markov path-dependent processes and Poisson-directed jump processes.

Journal ArticleDOI
TL;DR: A class of renewable-resource-allocation problems is studied for the processing of dynamically arriving tasks with deterministic deadlines, and can be solved, at least in principle, by using a stochastic dynamic programming (SDP) method.
Abstract: A class of renewable-resource-allocation problems is studied for the processing of dynamically arriving tasks with deterministic deadlines. The model presented explicitly considers time available, time required, resources available, resources required, stochastic arrivals of multiple types of tasks, importance of tasks, timeliness of processing, and accuracy of resource allocation. After state augmentation, the problem becomes a Markovian decision problem, and can be solved, at least in principle, by using a stochastic dynamic programming (SDP) method. Effects of key system parameters on optimal decisions are investigated and analyzed through numerical examples. >

BookDOI
01 Jan 1989
TL;DR: Various aspects of the rainfall-runoff modeling process are scrutinized by use of probabilistic models, such that when combined, a stochastic integral equation results.
Abstract: The uncertainty in rainfall-runoff modeling predictions has become a topic of recent key interest. In this book, the uncertainty problem is approached by use of stochastic integral equations. Various aspects of the rainfall-runoff modeling process are scrutinized by use of probabilistic models, such that when combined, a stochastic integral equation results. Uncertainty in single even runoff estimates, as well as return frequency event outcomes are analyzed. Use of example problems demonstrate the application of stochastic integral equations in addition to explaining the underlying concepts. Computer program source code is also provided which can be used to solve both theoretical and real-world problems. The generous supply of chapter problems enables the book to be used as an applied textbook in stochastic integrals.