THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

List of Figures. List of Tables. Preface. Foreword. 1. Basic Concepts. 2. Evolutionary Algorithm MOP Approaches. 3. MOEA Test Suites. 4. MOEA Testing and Analysis. 5. MOEA Theory and Issues. 3. MOEA Theoretical Issues. 6. Applications. 7. MOEA Parallelization. 8. Multi-Criteria Decision Making. 9. Special Topics. 10. Epilog. Appendix A: MOEA Classification and Technique Analysis. Appendix B: MOPs in the Literature. Appendix C: Ptrue & PFtrue for Selected Numeric MOPs. Appendix D: Ptrue & PFtrue for Side-Constrained MOPs. Appendix E: MOEA Software Availability. Appendix F: MOEA-Related Information. Index. References.

/pdf/evolutionary-algorithms-for-solving-multi-objective-problems-2txxk4mqci.pdf

Evolutionary algorithms for solving multi-objective problems

Economic Control of Quality of Manufactured Product.

The use of statistical techniques to build approximations of expensive computer analysis codes pervades much of today’s engineering design. These statistical approximations, or metamodels, are used to replace the actual expensive computer analyses, facilitating multidisciplinary, multiobjective optimization and concept exploration. In this paper, we review several of these techniques, including design of experiments, response surface methodology, Taguchi methods, neural networks, inductive learning and kriging. We survey their existing application in engineering design, and then address the dangers of applying traditional statistical techniques to approximate deterministic computer analysis codes. We conclude with recommendations for the appropriate use of statistical approximation techniques in given situations, and how common pitfalls can be avoided.

/pdf/metamodels-for-computer-based-engineering-design-survey-and-382x1fki1s.pdf

Metamodels for Computer-Based Engineering Design: Survey and Recommendations

Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems

This paper reviews statistical methods for analyzing output data from computer simulations. First, it focuses on the estimation of steady-state system parameters. The estimation techniques include the replication/deletion approach, the regenerative method, the batch means method, and methods based on standardized time series. Second, it reviews recent statistical procedures to find the best system among a set of competing alternatives.

Proceedings of the 2000 winter simulation conference

In many scientific and engineering domains, it is common to analyze and simulate complex physical systems using mathematical models. Although computing resources continue to increase in power and speed, computer simulation and analysis codes continue to grow in complexity and remain computationally expensive, limiting their use in design and optimization. Consequently, many researchers have developed different metamodeling strategies to create inexpensive approximations of computationally expensive computer simulations. These approximations introduce a new element of uncertainty during design optimization, and there is a need to develop efficient methods to assess metamodel validity. We investigate computationally inexpensive assessment methods for metamodel validation based on leave-k-out cross validation and develop guidelines for selecting k for different types of metamodels. Based on the results from two sets of test problems, k = 1 is recommended for leave-k-out cross validation of low-order polynomial and radial basis function metamodels, whereas k=0.1N or N is recommended for kriging metamodels, where N is the number of sample points used to construct the metamodel.

Computationally Inexpensive Metamodel Assessment Strategies

In this paper, we provide a review of statistical methods that are useful in conducting computer experiments. Our focus is on the task of metamodeling, which is driven by the goal of optimizing a complex system via a deterministic simulation model. However, we also mention the case of a stochastic simulation, and examples of both cases are discussed. The organization of our review first presents several engineering applications, it then describes approaches for the two primary tasks of metamodeling: (i) selecting an experimental design; and (ii) fitting a statistical model. Seven statistical modeling methods are included. Both classical and newer experimental designs are discussed. Finally, our own computational study tests the various metamodeling options on two two-dimensional response surfaces and one ten-dimensional surface.

A review on design, modeling and applications of computer experiments

Simulation models allow the user to understand system performance and assist in behavior prediction, to support system diagnostics and design. Iterative optimization methods are often used in conjunction with engineering simulation models to search for designs with desired properties. These optimization methods can be difficult to employ with a discrete-event simulation, due to the stochastic nature of the response(s) and the potentially extensive run times. A metamodel, or model of the simulation model, simplifies the simulation optimization in two ways: the metamodel response is deterministic rather than stochastic, and the run times are generally much shorter than the original simulation. Metamodels based on first- or second-order polynomials generally provide good fit only locally, and so a series of metamodels are fit as the optimization progresses. Other classes of metamodels can provide good global fit; in these cases one can fit a (global) metamodel once, at the start of the optimization, and use it to find a design that will meet the optimality criteria. Both approaches are discussed in this chapter and illustrated with an example.

Chapter 18 Metamodel-Based Simulation Optimization

When the Nelder-Mead method is used to optimize the expected response of a stochastic system e.g., an output of a discrete-event simulation model, the simplex-resizing steps of the method introduce risks of inappropriate termination. We give analytical and empirical results describing the performance of Nelder-Mead when it is applied to a response function that incorporates an additive white-noise error, and we use these results to develop new modifications of Nelder-Mead that yield improved estimates of the optimal expected response. Compared to Nelder-Mead, the best performance was obtained by a modified method, RS + S9, in which a the best point in the simplex is reevaluated at each shrink, step and b the simplex is reduced by 10% rather than 50% at each shrink step. In a suite of 18 test problems that were adapted from the MINPACK collection of NETLIB, the expected response at the estimated optimal point obtained by RS + S9 had errors that averaged 15% less than at the original method's estimated optimal point, at an average cost of three times as many function evaluations. Two well-known existing modifications for stochastic responses, the n + 3-rule and the next-to-worst rule, were found to be inferior to the new modification RS + S9.

Russell R. Barton

Papers

Proceedings of the 2000 winter simulation conference

Computationally Inexpensive Metamodel Assessment Strategies

A review on design, modeling and applications of computer experiments

Chapter 18 Metamodel-Based Simulation Optimization

Nelder-Mead simplex modifications for simulation optimization