Posted Content•
Adaptive optimal allocation in stratified sampling methods
TL;DR: In this article, a stratified sampling algorithm is proposed in which the random drawings made in the strata to compute the expectation of interest are also used to adaptively modify the proportion of further drawings in each stratum.
Abstract: In this paper, we propose a stratified sampling algorithm in which the random drawings made in the strata to compute the expectation of interest are also used to adaptively modify the proportion of further drawings in each stratum. These proportions converge to the optimal allocation in terms of variance reduction. And our stratified estimator is asymptotically normal with asymptotic variance equal to the minimal one. Numerical experiments confirm the efficiency of our algorithm.
Citations
More filters
01 Jan 2013
TL;DR: This paper studies the practical issues involved in developing a real-world UIM-based performance metric and develops a prototype implementation which is used to evaluate a number of different artificial agents.
Abstract: The Universal Intelligence Measure is a recently proposed formal definition of intelligence. It is mathematically specified, extremely general, and captures the essence of many informal definitions of intelligence. It is based on Hutter’s Universal Artificial Intelligence theory, an extension of Ray Solomonoff’s pioneering work on universal induction. Since the Universal Intelligence Measure is only asymptotically computable, building a practical intelligence test from it is not straightforward. This paper studies the practical issues involved in developing a real-world UIM-based performance metric. Based on our investigation, we develop a prototype implementation which we use to evaluate a number of different artificial agents.
65 citations
Cites methods from "Adaptive optimal allocation in stra..."
...There are various algorithms for adaptive stratified sampling, however we have chosen the method developed by Étoré and Jourdain (2010) as they have derived the confidence intervals for the estimate of the mean, a feature we will use when reporting our results....
[...]
...There are various algorithms for adaptive stratified sampling, however we have chosen the method developed by Étoré and Jourdain (2010) as they have derived the confidence intervals for the estimate of the mean, a feature we will use when reporting our results....
[...]
05 Oct 2011
TL;DR: This paper describes two strategies based on pulling the arms proportionally to an upper-bound on their variance and derive regret bounds for these strategies and shows that the performance of these allocation strategies depends not only on the variances of the arms but also on the full shape of their distribution.
Abstract: In this paper, we study the problem of estimating the mean values of all the arms uniformly well in the multi-armed bandit setting. If the variances of the arms were known, one could design an optimal sampling strategy by pulling the arms proportionally to their variances. However, since the distributions are not known in advance, we need to design adaptive sampling strategies to select an arm at each round based on the previous observed samples. We describe two strategies based on pulling the arms proportionally to an upper-bound on their variance and derive regret bounds for these strategies. We show that the performance of these allocation strategies depends not only on the variances of the arms but also on the full shape of their distribution.
61 citations
Cites methods from "Adaptive optimal allocation in stra..."
..., 2005), sampling and Monte-Carlo methods (Étoré and Jourdain, 2010), and optimal experimental design (Fedorov, 1972, Chaudhuri and Mykland, 1995)....
[...]
TL;DR: This paper proposes estimating the Shapley value using a reinforcement learning algorithm that approximates optimal stratified sampling and applies this algorithm to a DR program that utilizes the SV for payments and quantifies the accuracy of the resulting estimates.
Abstract: Designing fair compensation mechanisms for demand response (DR) is challenging. This paper models the problem in a game theoretic setting and designs a payment distribution mechanism based on the Shapley value (SV). As exact computation of the SV is in general intractable, we propose estimating it using a reinforcement learning algorithm that approximates optimal stratified sampling. We apply this algorithm to a DR program that utilizes the SV for payments and quantify the accuracy of the resulting estimates.
60 citations
Cites methods from "Adaptive optimal allocation in stra..."
...We implemented a number of such functions (including the stepped function described in [22]) and found the most accurate to be the double sigmoid function...
[...]
TL;DR: This paper proposes an incremental algorithm that asymptotically achieves the same loss as an optimal rule and proves that the excess loss suffered by this algorithm, apart from logarithmic factors, scales as n^-^3^/^2, which is conjecture to be the optimal rate.
59 citations
Cites background or methods from "Adaptive optimal allocation in stra..."
...This has been demonstrated convincingly in a forthcoming related paper where Etore and Jourdain studied the utility of adapting the sampling proportions in stratified sampling [8]....
[...]
...The algorithm studied in [8] is the phase-based algorithm....
[...]
...(An algorithm along these lines have been described and analyzed by [8] in the context of stratified sampling....
[...]
12 Oct 2008
TL;DR: This paper proposes an incremental algorithm that asymptotically achieves the same loss as an optimal rule and proves that the excess loss suffered by this algorithm, apart from logarithmic factors, scales as ni¾?
Abstract: In this paper we consider the problem of actively learning the mean values of distributions associated with a finite number of options (arms) The algorithms can select which option to generate the next sample from in order to produce estimates with equally good precision for all the distributions When an algorithm uses sample means to estimate the unknown values then the optimal solution, assuming full knowledge of the distributions, is to sample each option proportional to its variance In this paper we propose an incremental algorithm that asymptotically achieves the same loss as an optimal rule We prove that the excess loss suffered by this algorithm, apart from logarithmic factors, scales as ni¾? 3/2, which we conjecture to be the optimal rate The performance of the algorithm is illustrated in a simple problem
58 citations
Cites background or methods from "Adaptive optimal allocation in stra..."
...The algorithm studied in [8] is the phase-based algorithm....
[...]
...This has been demonstrated convincingly in a very recent related paper where the authors studied the utility of adapting the sampling proportions in stratified sampling [8]....
[...]
...(An algorithm along these lines have been described and analyzed by [8] in the context of stratified sampling....
[...]
References
More filters
Book•
07 Aug 2003
TL;DR: This paper presents a meta-modelling procedure that automates the very labor-intensive and therefore time-heavy and therefore expensive and expensive process of manually computing random numbers and random Variables.
Abstract: Foundations.- Generating Random Numbers and Random Variables.- Generating Sample Paths.- Variance Reduction Techniques.- Quasi-Monte Carlo Methods.- Discretization Methods.- Estimating Sensitivities.- Pricing American Options.- Applications in Risk Management.- Appendices
3,568 citations
TL;DR: In this article, a variance reduction technique for Monte Carlo simulations of path-dependent options driven by high-dimensional Gaussian vectors is proposed, which combines importance sampling based on a change of drift with stratified sampling along a small number of key dimensions.
Abstract: This paper develops a variance reduction technique for Monte Carlo simulations of path-dependent options driven by high-dimensional Gaussian vectors. The method combines importance sampling based on a change of drift with stratified sampling along a small number of key dimensions. The change of drift is selected through a large deviations analysis and is shown to be optimal in an asymptotic sense. The drift selected has an interpretation as the path of the underlying state variables which maximizes the product of probability and payoff—the most important path. The directions used for stratified sampling are optimal for a quadratic approximation to the integrand or payoff function. Indeed, under differentiability assumptions our importance sampling method eliminates variability due to the linear part of the payoff function, and stratification eliminates much of the variability due to the quadratic part of the payoff. The two parts of the method are linked because the asymptotically optimal drift vector frequently provides a particularly effective direction for stratification. We illustrate the use of the method with path-dependent options, a stochastic volatility model, and interest rate derivatives. The method reveals novel features of the structure of their payoffs.
246 citations
"Adaptive optimal allocation in stra..." refers background or methods or result in this paper
...In the sequel we take the same direction u and the same strata as in Glasserman et al. (1999) , and discuss allocation....
[...]
...We wish to compare our results with the ones of Glasserman et al. (1999) ....
[...]
...The authors of Glasserman et al. (1999) then propose to use a stratified estimator of c = E fμ(X)....
[...]
...The most significant part of the paper ( Glasserman et al. 1999 ) is aimed at giving an asymptotical sense to this heuristic, using large deviations tools....
[...]
... Glasserman et al. (1999) is built in the following way....
[...]
Book•
03 Nov 2003
TL;DR: Series Preface*Preface*Introduction*Linear Programming*Nonlinear Programming* Approximation Techniques*Variational Problems and Dynamic Programming*Optimal Control*References*Index
Abstract: Series Preface*Preface*Introduction*Linear Programming*Nonlinear Programming* Approximation Techniques*Variational Problems and Dynamic Programming*Optimal Control*References*Index
114 citations
TL;DR: An adaptative variance reduction method for Monte Carlo simulations that uses importance sampling scheme based on a change of drift and develops two applications of the procedure for variance reduction in a Monte Carlo computation in finance and in reliability.
Abstract: In this article we propose an adaptative variance reduction method for Monte Carlo simulations. The method uses importance sampling scheme based on a change of drift. The change of drift is selected adaptatively through the Monte Carlo computation by using a suitable sequence of approximation. We state and prove theoretical results supporting the use of the method. We develop two applications of the procedure for variance reduction in a Monte Carlo computation in finance and in reliability.
105 citations
TL;DR: In this paper, variance reduction techniques for the estimation of quantiles of the output of a complex model with random input parameters are discussed, based on the use of a reduced model, such as a metamodel or a response surface.
Abstract: In this paper we propose and discuss variance reduction techniques for the estimation of quantiles of the output of a complex model with random input parameters. These techniques are based on the use of a reduced model, such as a metamodel or a response surface. The reduced model can be used as a control variate; or a rejection method can be implemented to sample the realizations of the input parameters in prescribed relevant strata; or the reduced model can be used to determine a good biased distribution of the input parameters for the implementation of an importance sampling strategy. The different strategies are analyzed and the asymptotic variances are computed, which shows the benefit of an adaptive controlled stratification method. This method is finally applied to a real example (computation of the peak cladding temperature during a large-break loss of coolant accident in a nuclear reactor).
69 citations