Surrogate-based Analysis and Optimization

Computation-intensive design problems are becoming increasingly common in manufacturing industries. The computation burden is often caused by expensive analysis and simulation processes in order to reach a comparable level of accuracy as physical testing data. To address such a challenge, approximation or metamodeling techniques are often used. Metamodeling techniques have been developed from many different disciplines including statistics, mathematics, computer science, and various engineering disciplines. These metamodels are initially developed as “surrogates” of the expensive simulation process in order to improve the overall computation efficiency. They are then found to be a valuable tool to support a wide scope of activities in modern engineering design, especially design optimization. This work reviews the state-of-the-art metamodel-based techniques from a practitioner’s perspective according to the role of metamodeling in supporting design optimization, including model approximation, design space exploration, problem formulation, and solving various types of optimization problems. Challenges and future development of metamodeling in support of engineering design is also analyzed and discussed.Copyright © 2006 by ASME

https://www.sfu.ca/~gwa5/index_files/Microsoft%20Word%20-%20ASME-Metamodeling-6-12-06.pdf

Review of Metamodeling Techniques in Support of Engineering Design Optimization

There has been increasing interest in the problem of building accurate data mining models over aggregate data, while protecting privacy at the level of individual records. One approach for this problem is to randomize the values in individual records, and only disclose the randomized values. The model is then built over the randomized data, after first compensating for the randomization (at the aggregate level). This approach is potentially vulnerable to privacy breaches: based on the distribution of the data, one may be able to learn with high confidence that some of the randomized records satisfy a specified property, even though privacy is preserved on average.In this paper, we present a new formulation of privacy breaches, together with a methodology, "amplification", for limiting them. Unlike earlier approaches, amplification makes it is possible to guarantee limits on privacy breaches without any knowledge of the distribution of the original data. We instantiate this methodology for the problem of mining association rules, and modify the algorithm from [9] to limit privacy breaches without knowledge of the data distribution. Next, we address the problem that the amount of randomization required to avoid privacy breaches (when mining association rules) results in very long transactions. By using pseudorandom generators and carefully choosing seeds such that the desired items from the original transaction are present in the randomized transaction, we can send just the seed instead of the transaction, resulting in a dramatic drop in communication and storage cost. Finally, we define new information measures that take privacy breaches into account when quantifying the amount of privacy preserved by randomization.

/pdf/limiting-privacy-breaches-in-privacy-preserving-data-mining-7gg43mj62d.pdf

Limiting privacy breaches in privacy preserving data mining

A universal framework is developed for constructing full-rate and full-diversity coherent space-time codes for systems with arbitrary numbers of transmit and receive antennas. The proposed framework combines space-time layering concepts with algebraic component codes optimized for single-input-single-output (SISO) channels. Each component code is assigned to a "thread" in the space-time matrix, allowing it thus full access to the channel spatial diversity in the absence of the other threads. Diophantine approximation theory is then used in order to make the different threads "transparent" to each other. Within this framework, a special class of signals which uses algebraic number-theoretic constellations as component codes is thoroughly investigated. The lattice structure of the proposed number-theoretic codes along with their minimal delay allow for polynomial complexity maximum-likelihood (ML) decoding using algorithms from lattice theory. Combining the design framework with the Cayley transform allows to construct full diversity differential and noncoherent space-time codes. The proposed framework subsumes many of the existing codes in the literature, extends naturally to time-selective and frequency-selective channels, and allows for more flexibility in the tradeoff between power efficiency, bandwidth efficiency, and receiver complexity. Simulation results that demonstrate the significant gains offered by the proposed codes are presented in certain representative scenarios.

Universal space-time coding

The integration of optimization methodologies with computational analyses/simulations has a profound impact on the product design. Such integration, however, faces multiple challenges. The most eminent challenges arise from high-dimensionality of problems, computationally-expensive analysis/simulation, and unknown function properties (i.e., black-box functions). The merger of these three challenges severely aggravates the difficulty and becomes a major hurdle for design optimization. This paper provides a survey on related modeling and optimization strategies that may help to solve High-dimensional, Expensive (computationally), Black-box (HEB) problems. The survey screens out 207 references including multiple historical reviews on relevant subjects from more than 1,000 papers in a variety of disciplines. This survey has been performed in three areas: strategies tackling high-dimensionality of problems, model approximation techniques, and direct optimization strategies for computationally-expensive black-box functions and promising ideas behind non-gradient optimization algorithms. Major contributions in each area are discussed and presented in an organized manner. The survey exposes that direct modeling and optimization strategies to address HEB problems are scarce and sporadic, partially due to the difficulty of the problem itself. Moreover, it is revealed that current modeling research tends to focus on sampling and modeling techniques themselves and neglect studying and taking the advantages of characteristics of the underlying expensive functions. Based on the survey results, two promising approaches are identified to solve HEB problems. Directions for future research are also discussed.

/pdf/survey-of-modeling-and-optimization-strategies-to-solve-high-1tg25zsec9.pdf

Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions

1 Introduction.- 1.1 Problems.- 2 Rao's Inequalities and Improvements.- 2.1 Introduction.- 2.2 Rao's Inequalities.- 2.3 Improvements on Rao's Bounds for Strength 2 and 3.- 2.4 Improvements on Rao's Bounds for Arrays of Index Unity.- 2.5 Orthogonal Arrays with Two Levels.- 2.6 Concluding Remarks.- 2.7 Notes on Chapter 2.- 2.8 Problems.- 3 Orthogonal Arrays and Galois Fields.- 3.1 Introduction.- 3.2 Bush's Construction.- 3.3 Addelman and Kempthorne's Construction.- 3.4 The Rao-Hamming Construction.- 3.5 Conditions for a Matrix.- 3.6 Concluding Remarks.- 3.7 Problems.- 4 Orthogonal Arrays and Error-Correcting Codes.- 4.1 An Introduction to Error-Correcting Codes.- 4.2 Linear Codes.- 4.3 Linear Codes and Linear Orthogonal Arrays.- 4.4 Weight Enumerators and Delsarte's Theorem.- 4.5 The Linear Programming Bound.- 4.6 Concluding Remarks.- 4.7 Notes on Chapter 4.- 4.8 Problems.- 5 Construction of Orthogonal Arrays from Codes.- 5.1 Extending a Code by Adding More Coordinates.- 5.2 Cyclic Codes.- 5.3 The Rao-Hamming Construction Revisited.- 5.4 BCH Codes.- 5.5 Reed-Solomon Codes.- 5.6 MDS Codes and Orthogonal Arrays of Index Unity.- 5.7 Quadratic Residue and Golay Codes.- 5.8 Reed-Muller Codes.- 5.9 Codes from Finite Geometries.- 5.10 Nordstrom-Robinson and Related Codes.- 5.11 Examples of Binary Codes and Orthogonal Arrays.- 5.12 Examples of Ternary Codes and Orthogonal Arrays.- 5.13 Examples of Quaternary Codes and Orthogonal Arrays.- 5.14 Notes on Chapter 5.- 5.15 Problems.- 6 Orthogonal Arrays and Difference Schemes.- 6.1 Difference Schemes.- 6.2 Orthogonal Arrays Via Difference Schemes.- 6.3 Bose and Bush's Recursive Construction.- 6.4 Difference Schemes of Index 2.- 6.5 Generalizations and Variations.- 6.6 Concluding Remarks.- 6.7 Notes on Chapter 6.- 6.8 Problems.- 7 Orthogonal Arrays and Hadamard Matrices.- 7.1 Introduction.- 7.2 Basic Properties of Hadamard Matrices.- 7.3 The Connection Between Hadamard Matrices and Orthogonal Arrays.- 7.4 Constructions for Hadamard Matrices.- 7.5 Hadamard Matrices of Orders up to 200.- 7.6 Notes on Chapter 7.- 7.7 Problems.- 8 Orthogonal Arrays and Latin Squares.- 8.1 Latin Squares and Orthogonal Latin Squares.- 8.2 Frequency Squares and Orthogonal Frequency Squares.- 8.3 Orthogonal Arrays from Pairwise Orthogonal Latin Squares.- 8.4 Concluding Remarks.- 8.5 Problems.- 9 Mixed Orthogonal Arrays.- 9.1 Introduction.- 9.2 The Rao Inequalities for Mixed Orthogonal Arrays.- 9.3 Constructing Mixed Orthogonal Arrays.- 9.4 Further Constructions.- 9.5 Notes on Chapter 9.- 9.6 Problems.- 10 Further Constructions and Related Structures.- 10.1 Constructions Inspired by Coding Theory.- 10.2 The Juxtaposition Construction.- 10.3 The (u, u + ?) Construction.- 10.4 Construction X4.- 10.5 Orthogonal Arrays from Union of Translates of a Linear Code.- 10.6 Bounds on Large Orthogonal Arrays.- 10.7 Compound Orthogonal Arrays.- 10.8 Orthogonal Multi-Arrays.- 10.9 Transversal Designs, Resilient Functions and Nets.- 10.10 Schematic Orthogonal Arrays.- 10.11 Problems.- 11 Statistical Application of Orthogonal Arrays.- 11.1 Factorial Experiments.- 11.2 Notation and Terminology.- 11.3 Factorial Effects.- 11.4 Analysis of Experiments Based on Orthogonal Arrays.- 11.5 Two-Level Fractional Factorials with a Defining Relation.- 11.6 Blocking for a 2k-n Fractional Factorial.- 11.7 Orthogonal Main-Effects Plans and Orthogonal Arrays.- 11.8 Robust Design.- 11.9 Other Types of Designs.- 11.10 Notes on Chapter 11.- 11.11 Problems.- 12 Tables of Orthogonal Arrays.- 12.1 Tables of Orthogonal Arrays of Minimal Index.- 12.2 Description of Tables 12.1?12.3.- 12.3 Index Tables.- 12.4 If No Suitable Orthogonal Array Is Available.- 12.5 Connections with Other Structures.- 12.6 Other Tables.- Appendix A: Galois Fields.- A.1 Definition of a Field.- A.2 The Construction of Galois Fields.- A.3 The Existence of Galois Fields.- A.4 Quadratic Residues in Galois Fields.- A.5 Problems.- Author Index.

Lih-Yuan Deng

Papers

Orthogonal Arrays: Theory and Applications