The paper proposes a novel upper confidence bound (UCB) procedure for identifying the arm with the largest mean in a multi-armed bandit game in the fixed confidence setting using a small number of total samples. The procedure cannot be improved in the sense that the number of samples required to identify the best arm is within a constant factor of a lower bound based on the law of the iterated logarithm (LIL). Inspired by the LIL, we construct our confidence bounds to explicitly account for the infinite time horizon of the algorithm. In addition, by using a novel stopping time for the algorithm we avoid a union bound over the arms that has been observed in other UCBtype algorithms. We prove that the algorithm is optimal up to constants and also show through simulations that it provides superior performance with respect to the state-of-the-art.

/pdf/lil-ucb-an-optimal-exploration-algorithm-for-multi-armed-1l53749f1b.pdf

lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Bandits

1 Extension and applications of an inequality of Ville and Wald Let x 1… be a sequence of random variables with a specified joint probability distribution P We shall give a method for obtaining probability inequalities and related limit theorems concerning the behavior of the entire sequence of x’s

https://projecteuclid.org/download/pdf_1/euclid.aoms/1177696786

Statistical Methods Related to the Law of the Iterated Logarithm

1. Introduction and summary. Let W(t) denote a standard Wiener process for 0 ≦ t 0 (or for some t > 0) for a certain class of functions g(t), including functions which are ~ (2t log log t)½ as y → ∞. We also prove an invariance theorem which states that this probability is the limit as m → ∞ of the probability that S n ≦m ½ g(n/m) for some n ≦ τm (or for some n ≦ 1), where S n is the nth partial sum of any sequence x 1, x 2, … of independent and identically distributed (i.i.d.) random variables with mean 0 and variance 1.

/pdf/boundary-crossing-probabilities-for-the-wiener-process-and-514jzzq13z.pdf

Boundary Crossing Probabilities for the Wiener Process and Sample Sums

A confidence sequence is a sequence of confidence intervals that is uniformly valid over an unbounded time horizon. Our work develops confidence sequences whose widths go to zero, with nonasymptotic coverage guarantees under nonparametric conditions. We draw connections between the Cramer–Chernoff method for exponential concentration, the law of the iterated logarithm (LIL) and the sequential probability ratio test—our confidence sequences are time-uniform extensions of the first; provide tight, nonasymptotic characterizations of the second; and generalize the third to nonparametric settings, including sub-Gaussian and Bernstein conditions, self-normalized processes and matrix martingales. We illustrate the generality of our proof techniques by deriving an empirical-Bernstein bound growing at a LIL rate, as well as a novel upper LIL for the maximum eigenvalue of a sum of random matrices. Finally, we apply our methods to covariance matrix estimation and to estimation of sample average treatment effect under the Neyman–Rubin potential outcomes model.

/pdf/time-uniform-nonparametric-nonasymptotic-confidence-4kzk8lnxqo.pdf

Time-uniform, nonparametric, nonasymptotic confidence sequences

We propose a general method for constructing confidence sets and hypothesis tests that have finite-sample guarantees without regularity conditions We refer to such procedures as “universal” The method is very simple and is based on a modified version of the usual likelihood-ratio statistic that we call “the split likelihood-ratio test” (split LRT) statistic The (limiting) null distribution of the classical likelihood-ratio statistic is often intractable when used to test composite null hypotheses in irregular statistical models Our method is especially appealing for statistical inference in these complex setups The method we suggest works for any parametric model and also for some nonparametric models, as long as computing a maximum-likelihood estimator (MLE) is feasible under the null Canonical examples arise in mixture modeling and shape-constrained inference, for which constructing tests and confidence sets has been notoriously difficult We also develop various extensions of our basic methods We show that in settings when computing the MLE is hard, for the purpose of constructing valid tests and intervals, it is sufficient to upper bound the maximum likelihood We investigate some conditions under which our methods yield valid inferences under model misspecification Further, the split LRT can be used with profile likelihoods to deal with nuisance parameters, and it can also be run sequentially to yield anytime-valid P values and confidence sequences Finally, when combined with the method of sieves, it can be used to perform model selection with nested model classes

https://www.pnas.org/content/pnas/117/29/16880.full.pdf

Universal inference

Confidence sequences for mean, variance, and median.

1. Introduction—Let x,x 1, x 2 … be a sequence of independent, identically distributed random variables with mean 0, variance 1, and moment generating function ϕ(t) = E(etz) finite in some neighborhood of t= 0, and put S n = x 1, + … + x n, \(\bar x\) n = S n/n. For any sequence of positive constants a n, n ≥ 1, let P m = P(|\(\bar x\) n| ≥ a n for some n ≥ m).

D. A. Darling

Papers

Confidence sequences for mean, variance, and median.

Iterated logarithm inequalities.

Some nonparametric sequential tests with power one.

Inequalities for the sequence of sample means.