Power grids, flexible manufacturing, cellular communications: interconnectedness has consequences. This remarkable book gives the tools and philosophy you need to build network models detailed enough to capture essential dynamics but simple enough to expose the structure of effective control solutions and to clarify analysis. Core chapters assume only exposure to stochastic processes and linear algebra at the undergraduate level; later chapters are for advanced graduate students and researchers/practitioners. This gradual development bridges classical theory with the state-of-the-art. The workload model that is the basis of traditional analysis of the single queue becomes a foundation for workload relaxations used in the treatment of complex networks. Lyapunov functions and dynamic programming equations lead to the celebrated MaxWeight policy along with many generalizations. Other topics include methods for synthesizing hedging and safety stocks, stability theory for networks, and techniques for accelerated simulation. Examples and figures throughout make ideas concrete. Solutions to end-of-chapter exercises available on a companion website.

http://www.meyn.ece.ufl.edu/archive/spm_files/Presentations/Atlanta05.pdf

Control Techniques for Complex Networks

https://link.springer.com/content/pdf/10.1057%2Fjors.1994..pdf

Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method

Proceedings of the 2008 winter simulation conference

Summary. A general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible Markov chain Monte Carlo samplers. We propose the use of a specific class of functions as control variates, and we introduce a new consistent estimator for the values of the coefficients of the optimal linear combination of these functions. For a specific Markov chain Monte Carlo scenario, the form and proposed construction of the control variates is shown to provide an exact solution of the associated Poisson equation. This implies that the estimation variance in this case (in the central limit theorem regime) is exactly zero. The new estimator is derived from a novel, finite dimensional, explicit representation for the optimal coefficients.The resulting variance reduction methodology is primarily (though certainly not exclusively) applicable when the simulated data are generated by a random-scan Gibbs sampler. Markov chain Monte Carlo examples of Bayesian inference problems demonstrate that the corresponding reduction in the estimation variance is significant, and that in some cases it can be quite dramatic. Extensions of this methodology are discussed and simulation examples are presented illustrating the utility of the methods proposed. All methodological and asymptotic arguments are rigorously justified under essentially minimal conditions.

/pdf/control-variates-for-estimation-based-on-reversible-markov-3vm8vwhghe.pdf

Control variates for estimation based on reversible Markov chain Monte Carlo samplers

A methodology is given for modeling the dynamics of discrete-event stochastic systems as optimization problems. The intent is to provide a means to utilize the rich mathematical theory and algorithms of optimization in the study of this important class of systems. A procedure for mapping a simulation event relationship graph into a mixed-integer program is presented, along with examples of queueing networks and manufacturing systems that illustrate the approach. Several potential applications are examined, including automatic constraint generation for optimal resource scheduling, representations of max-plus algebra models for queueing system dynamics, response gradient estimation, and an unconventional technique for simulating queueing systems using virtual resources that are identified from the optimization models for these systems.

Optimization Models of Discrete-Event System Dynamics

We present a review of methods for simulation optimization In particular, we focus on gradient-based techniques for continuous optimization We demonstrate the main concepts using as an example the multidimensional newsvendor problem We also discuss mathematical techniques and results that are useful in verifying and analyzing the simulation optimization procedures

Gradient-based simulation optimization

Adaptive Monte Carlo methods are simulation efficiency improvement techniques designed to adaptively tune simulation estimators. Most of the work on adaptive Monte Carlo methods has been devoted to adaptively tuning importance sampling schemes. We instead focus on adaptive methods based on control variate schemes. We introduce two adaptive control variate methods, and develop their asymptotic properties. The first method uses stochastic approximation to adaptively tune control variate estimators. It is easy to implement, but it requires some nontrivial tuning of parameters. The second method is based on sample average approximation. Tuning is no longer required, but it can be computationally expensive. Numerical results for the pricing of barrier options are presented to demonstrate the methods.

/pdf/adaptive-control-variates-for-finite-horizon-simulation-1fsg48h4te.pdf

Adaptive Control Variates for Finite-Horizon Simulation

Adaptive Monte Carlo methods are specialized Monte Carlo simulation techniques where the methods are adaptively tuned as the simulation progresses. The primary focus of such techniques has been in adaptively tuning importance sampling distributions to reduce the variance of an estimator. We instead focus on adaptive control variate schemes, developing asymptotic theory for the performance of two adaptive control variate estimators. The first estimator is based on a stochastic approximation scheme for identifying the optimal choice of control variate. It is easily implemented, but its performance is sensitive to certain tuning parameters, the selection of which is nontrivial. The second estimator uses a sample average approximation approach. It has the advantage that it does not require any tuning parameters, but it can be computationally expensive and requires the availability of nonlinear optimization software.

/pdf/adaptive-control-variates-4nbpmwigsh.pdf

Adaptive control variates

We assume the existence of a parameterized family of control variates that could be used in a regenerative steady-state simulation. We show how such controls can be generated in the Markov-process setting, discuss the optimization problem of searching for a good choice of parameterization, and develop a strong law and central limit theorem for the resulting estimator.

/pdf/non-linear-control-variates-for-regenerative-steady-state-1ywslin5gs.pdf

Sujin Kim

Papers

Gradient-based simulation optimization

Adaptive Control Variates for Finite-Horizon Simulation

Adaptive control variates

Non-linear control variates for regenerative steady-state simulation