Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of less than 20 dimensions, and tolerates stochastic noise in function evaluations. It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression, and then uses an acquisition function defined from this surrogate to decide where to sample. In this tutorial, we describe how Bayesian optimization works, including Gaussian process regression and three common acquisition functions: expected improvement, entropy search, and knowledge gradient. We then discuss more advanced techniques, including running multiple function evaluations in parallel, multi-fidelity and multi-information source optimization, expensive-to-evaluate constraints, random environmental conditions, multi-task Bayesian optimization, and the inclusion of derivative information. We conclude with a discussion of Bayesian optimization software and future research directions in the field. Within our tutorial material we provide a generalization of expected improvement to noisy evaluations, beyond the noise-free setting where it is more commonly applied. This generalization is justified by a formal decision-theoretic argument, standing in contrast to previous ad hoc modifications.

A Tutorial on Bayesian Optimization

An intelligent trial-and-error learning algorithm is presented that allows robots to adapt in minutes to compensate for a wide variety of types of damage. Autonomous mobile robots would be extremely useful in remote or hostile environments such as space, deep oceans or disaster areas. An outstanding challenge is to make such robots able to recover after damage. Jean-Baptiste Mouret and colleagues have developed a machine learning algorithm that enables damaged robots to quickly regain their ability to perform tasks. When they sustain damage — such as broken or even missing legs — the robots adopt an intelligent trial-and-error approach, trying out possible behaviours that they calculate to be potentially high-performing. After a handful of such experiments they discover, in less than two minutes, a compensatory behaviour that works in spite of the damage. Robots have transformed many industries, most notably manufacturing1, and have the power to deliver tremendous benefits to society, such as in search and rescue2, disaster response3, health care4 and transportation5. They are also invaluable tools for scientific exploration in environments inaccessible to humans, from distant planets6 to deep oceans7. A major obstacle to their widespread adoption in more complex environments outside factories is their fragility6,8. Whereas animals can quickly adapt to injuries, current robots cannot ‘think outside the box’ to find a compensatory behaviour when they are damaged: they are limited to their pre-specified self-sensing abilities, can diagnose only anticipated failure modes9, and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots6,8. A promising approach to reducing robot fragility involves having robots learn appropriate behaviours in response to damage10,11, but current techniques are slow even with small, constrained search spaces12. Here we introduce an intelligent trial-and-error algorithm that allows robots to adapt to damage in less than two minutes in large search spaces without requiring self-diagnosis or pre-specified contingency plans. Before the robot is deployed, it uses a novel technique to create a detailed map of the space of high-performing behaviours. This map represents the robot’s prior knowledge about what behaviours it can perform and their value. When the robot is damaged, it uses this prior knowledge to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a behaviour that compensates for the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new algorithm will enable more robust, effective, autonomous robots, and may shed light on the principles that animals use to adapt to injury.

/pdf/robots-that-can-adapt-like-animals-21owdbvqr7.pdf

Robots that can adapt like animals

Theoretical approaches to brokerage and translation in development / David Mosse and David Lewis -- Aid policies and recipient strategies in Niger : why donors and recipients should not be compartmentalized into separate "worlds of knowledge" / Benedetta Rossi -- Resources, ideologies, and nationalism : the politics of development in Malaysia / Amity A. Doolittle -- Governing land, translating rights : the rural land plan in Benin / Pierre-Yves Le Meur -- Translating, interpreting, and practicing civil society in Vietnam : a tale of calculated misunderstandings / Oscar Salemink -- Brokering fair trade relations between coffee cooperatives and alternative trade organizations : a view from Costa Rica / Peter Luetchford -- Ethnographic research in a non-governmental organization : revealing strategic translations through an embedded tale / Wiebe Nauta -- Inside out : rationalizing practices and representations in agricultural development projects / Bina Desai -- "They can't mix like we can" : bracketing differences and the professionalization of NGOs in Nepal / Celayne Heaton Shrestha -- Rethinking the mechanics of the "anti-politics machine" / Tim Bending and Sergio Rosendo.

Development Brokers and Translators. The Ethnography of Aid and Agencies

We consider a Bayesian ranking and selection problem with independent normal rewards and a correlated multivariate normal belief on the mean values of these rewards. Because this formulation of the ranking and selection problem models dependence between alternatives' mean values, algorithms may use this dependence to perform efficiently even when the number of alternatives is very large. We propose a fully sequential sampling policy called the knowledge-gradient policy, which is provably optimal in some special cases and has bounded suboptimality in all others. We then demonstrate how this policy may be applied to efficiently maximize a continuous function on a continuous domain while constrained to a fixed number of noisy measurements.

/pdf/the-knowledge-gradient-policy-for-correlated-normal-beliefs-1m1rurejdp.pdf

The Knowledge-Gradient Policy for Correlated Normal Beliefs

Picking the right cloud configuration for recurring big data analytics jobs running in clouds is hard, because there can be tens of possible VM instance types and even more cluster sizes to pick from. Choosing poorly can significantly degrade performance and increase the cost to run a job by 2-3x on average, and as much as 12x in the worst-case. However, it is challenging to automatically identify the best configuration for a broad spectrum of applications and cloud configurations with low search cost. CherryPick is a system that leverages Bayesian Optimization to build performance models for various applications, and the models are just accurate enough to distinguish the best or close-to-the-best configuration from the rest with only a few test runs. Our experiments on five analytic applications in AWS EC2 show that CherryPick has a 45-90% chance to find optimal configurations, otherwise near-optimal, saving up to 75% search cost compared to existing solutions.

/pdf/cherrypick-adaptively-unearthing-the-best-cloud-21u26pl50w.pdf

Cherrypick: adaptively unearthing the best cloud configurations for big data analytics

1 Global optimization and the Bayesian approach.- 1.1 What is global optimization?.- 1.2 Advantages of the Bayesian approach to global optimization.- 2 The conditions of Bayesian optimality.- 2.1 Introduction.- 2.2 Reduction to dynamic programming equations.- 2.3 The existence of a measurable solution.- 2.4 The calculation of conditional expectations.- 2.5 The one-step approximation.- 2.6 The adaptive Bayesian approach.- 3 The axiomatic non-probabilistic justification of Bayesian optimality conditions.- 3.1 Introduction.- 3.2 The linearity of the loss function.- 3.3 The existence of the unique a priori probability corresponding to subjective preferences.- 3.4 Optimal method under uncertainty.- 3.5 Nonlinear loss functions.- 4 Stochastic models.- 4.1 Introduction.- 4.2 Sufficient convergence conditions.- 4.3 The Gaussian field.- 4.4 Homogeneous Wiener field.- 4.5 A case of noisy observations.- 4.6 Estimation of parameters from dependent observations.- 5 Bayesian methods for global optimization in the Gaussian case.- 5.1 The one-step approximation.- 5.2 Adaptive models.- 5.3 Extrapolation models.- 5.4 Maximum likelihood models.- 5.5 The comparison of algorithms.- 5.6 The Bayesian approach to global optimization with linear constraints.- 5.7 The Bayesian approach to global optimization with nonlinear constraints.- 5.8 The Bayesian approach to multi-objective optimization.- 5.9 Interactive procedures and the Bayesian approach to global optimization.- 5.10 The reduction of multi-dimensional data.- 5.11 The stopping rules.- 6 The analysis of structure and the simplification of the optimization problems.- 6.1 Introduction.- 6.2 Structural characteristics and the optimization problem.- 6.3 The estimation of structural characteristics.- 6.4 The estimation of a simplification error.- 6.5 Examples of the estimates.- 7 The Bayesian approach to local optimization.- 7.1 Introduction.- 7.2 The one-dimensional Bayesian model.- 7.3 Convergence of the local Bayesian algorithm.- 7.4 Generalization of a multi-dimensional case.- 7.5 Convergence in the multi-dimensional case.- 7.6 The local Bayesian algorithm.- 7.7 Results of computer simulation.- 8 The application of Bayesian methods.- 8.1 Introduction.- 8.2 The optimization of an electricity meter.- 8.3 The optimization of vibromotors.- 8.4 The optimization of a shock-absorber.- 8.5 The optimization of a magnetic beam deflection system.- 8.6 The optimization of small aperture coupling between a rectangular waveguide and a microstrip line.- 8.7 The maximization of LSI yield by optimization of parameters of differential amplifier functional blocks.- 8.8 Optimization of technology to avoid waste in the wet-etching of printed circuit boards in iron-copper-chloride solutions.- 8.9 The optimization of pigment compounds.- 8.10 The least square estimation of electrochemical adsorption using observations of the magnitude of electrode impedance.- 8.11 Estimation of parameters of the immunological model.- 8.12 The optimization of nonstationary queuing systems.- 8.13 The analysis of structure of the Steiner problem.- 8.14 The estimation of decision making by intuition on the example of the Steiner problem.- 9 Portable FORTRAN software for global optimization.- 9.1 Introduction.- 9.2 Parameters.- 9.3 Methods available.- 9.4 Common blocks.- 9.5 The function.- 9.6 The main program.- 9.7 The example of the main program.- 9.8 Description of routines.- 9.9 BAYES1, the global Bayesian method by Mockus.- 9.10 UNT, the global method of extrapolation type by Zilinskas.- 9.11 LPMIN, the global method of uniform search by Sobolj, Shaltenis and Dzemyda.- 9.12 GLOPT, the global method of clustering type by Torn.- 9.13 MIG1, the global method of Monte Carlo (uniform random search).- 9.14 MIG2, the modified version of MIG 1.- 9.15 EXTR, the global one-dimensional method by Zilinskas.- 9.16 MIVAR4, the local method of variable metrics by Tieshis.- 9.17 REQP, the local method of recursive quadratic programming by Biggs.- 9.18 FLEXI, the local simplex method by Nelder and Mead.- 9.19 LBAYES, the local Bayesian method by Mockus.- 9.20 ANAL1, the method of analysis by structure by Shaltenis.- 9.21 Portability routines.- References.- Appendix 1 The software for global optimization for IMB/PC/XT/AT and compatibles.- Appendix 2 How the global optimization software can improve the performance of your CAD system.- Appendix 3 Machine dependent constants of portable FORTRAN.

Jonas Mockus

Papers

Bayesian Approach to Global Optimization: Theory and Applications