We review the space-mapping (SM) technique and the SM-based surrogate (modeling) concept and their applications in engineering design optimization. For the first time, we present a mathematical motivation and place SM into the context of classical optimization. The aim of SM is to achieve a satisfactory solution with a minimal number of computationally expensive "fine" model evaluations. SM procedures iteratively update and optimize surrogates based on a fast physically based "coarse" model. Proposed approaches to SM-based optimization include the original algorithm, the Broyden-based aggressive SM algorithm, various trust-region approaches, neural SM, and implicit SM. Parameter extraction is an essential SM subproblem. It is used to align the surrogate (enhanced coarse model) with the fine model. Different approaches to enhance uniqueness are suggested, including the recent gradient parameter-extraction approach. Novel physical illustrations are presented, including the cheese-cutting and wedge-cutting problems. Significant practical applications are reviewed.

https://desi.iteso.mx/erayas/documents/CirOpt_course/Lectures/Space_Mapping/Bandler_04Jan_SM_State_of_the_Art.pdf

Space mapping: the state of the art

Networks are a fundamental tool for understanding and modeling complex systems in physics, biology, neuroscience, engineering, and social science. Many networks are known to exhibit rich, lower-order connectivity patterns that can be captured at the level of individual nodes and edges. However, higher-order organization of complex networks—at the level of small network subgraphs—remains largely unknown. Here, we develop a generalized framework for clustering networks on the basis of higher-order connectivity patterns. This framework provides mathematical guarantees on the optimality of obtained clusters and scales to networks with billions of edges. The framework reveals higher-order organization in a number of networks, including information propagation units in neuronal networks and hub structure in transportation networks. Results show that networks exhibit rich higher-order organizational structures that are exposed by clustering based on higher-order connectivity patterns.

/pdf/higher-order-organization-of-complex-networks-3n5pj1c6nh.pdf

Higher-order organization of complex networks

[1] Surrogate modeling, also called metamodeling, has evolved and been extensively used over the past decades. A wide variety of methods and tools have been introduced for surrogate modeling aiming to develop and utilize computationally more efficient surrogates of high-fidelity models mostly in optimization frameworks. This paper reviews, analyzes, and categorizes research efforts on surrogate modeling and applications with an emphasis on the research accomplished in the water resources field. The review analyzes 48 references on surrogate modeling arising from water resources and also screens out more than 100 references from the broader research community. Two broad families of surrogates namely response surface surrogates, which are statistical or empirical data-driven models emulating the high-fidelity model responses, and lower-fidelity physically based surrogates, which are simplified models of the original system, are detailed in this paper. Taxonomies on surrogate modeling frameworks, practical details, advances, challenges, and limitations are outlined. Important observations and some guidance for surrogate modeling decisions are provided along with a list of important future research directions that would benefit the common sampling and search (optimization) analyses found in water resources.

https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2011WR011527

Review of surrogate modeling in water resources

The integration of optimization methodologies with computational analyses/simulations has a profound impact on the product design. Such integration, however, faces multiple challenges. The most eminent challenges arise from high-dimensionality of problems, computationally-expensive analysis/simulation, and unknown function properties (i.e., black-box functions). The merger of these three challenges severely aggravates the difficulty and becomes a major hurdle for design optimization. This paper provides a survey on related modeling and optimization strategies that may help to solve High-dimensional, Expensive (computationally), Black-box (HEB) problems. The survey screens out 207 references including multiple historical reviews on relevant subjects from more than 1,000 papers in a variety of disciplines. This survey has been performed in three areas: strategies tackling high-dimensionality of problems, model approximation techniques, and direct optimization strategies for computationally-expensive black-box functions and promising ideas behind non-gradient optimization algorithms. Major contributions in each area are discussed and presented in an organized manner. The survey exposes that direct modeling and optimization strategies to address HEB problems are scarce and sporadic, partially due to the difficulty of the problem itself. Moreover, it is revealed that current modeling research tends to focus on sampling and modeling techniques themselves and neglect studying and taking the advantages of characteristics of the underlying expensive functions. Based on the survey results, two promising approaches are identified to solve HEB problems. Directions for future research are also discussed.

/pdf/survey-of-modeling-and-optimization-strategies-to-solve-high-1tg25zsec9.pdf

Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions

The use of metamodeling techniques in the design and analysis of computer experiments has progressed remarkably in the past 25 years, but how far has the field really come? This is the question addressed in this paper, namely, the extent to which the use of metamodeling techniques in multidisciplinary design optimization have evolved in the 25 years since the seminal paper on design and analysis of computer experiments by Sacks et al. (“Design and Analysis of Computer Experiments,” Statistical Science, Vol. 4, No. 4, 1989, pp. 409–435). Rather than a technical review of the entire body of metamodeling literature, the focus is on the evolution and motivation for advancements in metamodeling with some discussion on the research itself; not surprisingly, much of the current research motivation is the same as it was in the past. Based on current research thrusts in the field, multifidelity approximations and ensembles (i.e., sets) of metamodels, as well as the availability of metamodels within commercial soft...

Metamodeling in Multidisciplinary Design Optimization: How Far Have We Really Come?

This paper presents a comprehensive approach to engineering design optimization exploiting space mapping (SM). The algorithms employ input SM and a new generalization of implicit SM to minimize the misalignment between the coarse and fine models of the optimized object over a region of interest. Output SM ensures the matching of responses and first-order derivatives between the mapped coarse model and the fine model at the current iteration point in the optimization process. We provide theoretical results that show the importance of the explicit use of sensitivity information to the convergence properties of our family of algorithms. Our algorithm is demonstrated on the optimization of a microstrip bandpass filter, a bandpass filter with double-coupled resonators, and a seven-section impedance transformer. We describe the novel user-oriented software package SMF that implements the new family of SM optimization algorithms

/pdf/a-space-mapping-framework-for-engineering-optimization-246q86h4sc.pdf

A Space-Mapping Framework for Engineering Optimization&#8212;Theory and Implementation

Output space mapping is a technique introduced to enhance the robustness of the space-mapping optimization process in case the space-mapped coarse model cannot provide sufficient matching with the fine model. The technique often works very well; however, in some cases it fails. Especially in the microwave area where the typical model response (e.g., |S 21|) is a highly nonlinear function of the free parameter (e.g., frequency), the output space-mapping correction term may actually increase the mismatch between the surrogate and fine models for points other than the one at which the term was calculated, as in the surrogate model optimization process. In this paper, an adaptive response correction scheme is presented to work in conjunction with space-mapping optimization algorithms. This technique is designed to alleviate the difficulties of the standard output space mapping by adaptive adjustment of the response correction term according to the changes of the space-mapped coarse model response. Examples indicate the robustness of our approach.

/pdf/space-mapping-with-adaptive-response-correction-for-3pqnx33uoq.pdf

Space Mapping With Adaptive Response Correction for Microwave Design Optimization

One of the central issues in space mapping optimization is the quality of the underlying coarse models and surrogates. Whether a coarse model is sufficiently similar to the fine model may be critical to the performance of the space mapping optimization algorithm and a poor coarse model may result in lack of convergence. Although similarity requirements can be expressed with proper analytical conditions, it is difficult to verify such conditions beforehand for real-world engineering optimization problems. In this paper, we provide methods of assessing the quality of coarse/surrogate models. These methods can be used to predict whether a given model might be successfully used in space mapping optimization, to compare the quality of different coarse models, or to choose the proper type of space mapping which would be suitable to a given engineering design problem. Our quality estimation methods are derived from convergence results for space mapping algorithms. We provide illustrations and several practical application examples.

/pdf/quality-assessment-of-coarse-models-and-surrogates-for-space-lxyi2lfixf.pdf

Quality assessment of coarse models and surrogates for space mapping optimization

The space mapping technique is intended for optimization of engineering models which involve very expensive function evaluations. It is assumed that two different models of the same physical system are available: Besides the expensive model of primary interest (denoted the fine model), access to a cheaper (coarse) model is assumed which may be less accurate. The main idea of the space mapping technique is to use the coarse model to gain information about the fine model, and to apply this in the search for an optimal solution of the latter. Thus the technique iteratively establishes a mapping between the parameters of the two models which relate similar model responses. Having this mapping, most of the model evaluations can be directed to the fast coarse model. In many cases this technique quickly provides an approximate optimal solution to the fine model that is suffi- ciently accurate for engineering purposes. Thus the space mapping technique may be considered a preprocessing technique that perhaps must be succeeded by use of classical optimization techniques. We present an automatic scheme which integrates the space mapping and classical techniques.

/pdf/an-introduction-to-the-space-mapping-technique-3751s3z8no.pdf

An Introduction to the Space Mapping Technique

A powerful new Aggressive Space Mapping (ASM) optimization algorithm is presented It draws upon recent developments in both surrogate-based optimization and microwave device neuromodeling Our surrogate formulation (new to microwave engineering) exploits, in a novel way, a linear frequency-space mapping This is a powerful approach to severe response misalignments

/pdf/space-mapping-optimization-of-microwave-circuits-exploiting-493a8vl1lw.pdf

Kaj Madsen

Papers

A Space-Mapping Framework for Engineering Optimization—Theory and Implementation

Space Mapping With Adaptive Response Correction for Microwave Design Optimization

Quality assessment of coarse models and surrogates for space mapping optimization

An Introduction to the Space Mapping Technique

Space mapping optimization of microwave circuits exploiting surrogate models

Kaj Madsen

Papers

A Space-Mapping Framework for Engineering Optimization&#8212;Theory and Implementation

Space Mapping With Adaptive Response Correction for Microwave Design Optimization

Quality assessment of coarse models and surrogates for space mapping optimization

An Introduction to the Space Mapping Technique

Space mapping optimization of microwave circuits exploiting surrogate models

A Space-Mapping Framework for Engineering Optimization—Theory and Implementation