Sequential sampling problems arise in stochastic simulation and many other applications. Sampling is used to infer the unknown performance of several alternatives before one alternative is selected as best. This paper presents new economically motivated fully sequential sampling procedures to solve such problems, called economics of selection procedures. The optimal procedure is derived for comparing a known standard with one alternative whose unknown reward is inferred with sampling. That result motivates heuristics when multiple alternatives have unknown rewards. The resulting procedures are more effective in numerical experiments than any previously proposed procedure of which we are aware and are easily implemented. The key driver of the improvement is the use of dynamic programming to model sequential sampling as an option to learn before selecting an alternative. It accounts for the expected benefit of adaptive stopping policies for sampling, rather than of one-stage policies, as is common in the literature.

This paper was accepted by Assaf Zeevi, stochastic models and simulation.

/pdf/sequential-sampling-with-economics-of-selection-procedures-ak9bgkai78.pdf

Sequential Sampling with Economics of Selection Procedures

Contextual bandits serve as a fundamental model for many sequential decision making tasks. The most popular theoretically justified approaches are based on the optimism principle. While these algorithms can be practical, they are known to be suboptimal asymptotically. On the other hand, existing asymptotically optimal algorithms for this problem do not exploit the linear structure in an optimal way and suffer from lower-order terms that dominate the regret in all practically interesting regimes. We start to bridge the gap by designing an algorithm that is asymptotically optimal and has good finite-time empirical performance. At the same time, we make connections to the recent literature on when exploration-free methods are effective. Indeed, if the distribution of contexts is well behaved, then our algorithm acts mostly greedily and enjoys sub-logarithmic regret. Furthermore, our approach is adaptive in the sense that it automatically detects the nice case. Numerical results demonstrate significant regret reductions by our method relative to several baselines.

Adaptive Exploration in Linear Contextual Bandit

https://statweb.stanford.edu/~ckirby/lai/pubs/2013_BiomarkerGuided.pdf

Group sequential designs for developing and testing biomarker-guided personalized therapies in comparative effectiveness research.

Motivated by applications to confirmatory clinical trials for testing a new treatment against a placebo or active control when the new treatment has k possible treatment strategies (arms)-for example, k possible doses for a new drug-we develop an asymptotic theory for efficient outcome-adaptive randomization schemes and optimal stopping rules. Our approach consists of developing asymptotic lower bounds for the expected sample sizes from the k treatment arms and the control arm and using generalized sequential likelihood ratio procedures to achieve these bounds. Implementation details of our design and analysis and comparative simulation studies are also provided.

/pdf/efficient-adaptive-randomization-and-stopping-rules-in-multi-16g5qjw15f.pdf

Efficient Adaptive Randomization and Stopping Rules in Multi-arm Clinical Trials for Testing a New Treatment

In situations when many regimens are possible candidates for a large phase III study, but too few resources are available to evaluate each relative to the standard, conducting a multi-armed randomized selection trial is a useful strategy to remove inferior treatments from further consideration. When the study has a relatively quick endpoint such as an imaging-based lesion volume change in acute stroke patients, frequent interim monitoring of the trial is ethically and practically appealing to clinicians. In this article, I propose a class of sequential selection boundaries for multi-armed clinical trials, in which the objective is to select a treatment with a clinically significant improvement upon the control group, or to declare futility if no such treatment exists. The proposed boundaries are easy to implement in a blinded fashion, and can be applied on a flexible monitoring schedule in terms of calendar time. Design calibration with respect to prespecified levels of confidence is simple, and can be accomplished when the response rate of the control group is known only up to an interval. One of the proposed methods is applied to redesign a selection trial with an imaging endpoint in acute stroke patients, and is compared to an optimal two-stage design via simulations: The proposed method imposes smaller sample size on average than the two-stage design; this advantage is substantial when there is in fact a superior treatment to the control group.

Simple sequential boundaries for treatment selection in multi-armed randomized clinical trials with a control.

Given k ≥ 2 populations from an exponential family, we consider herein the problem of efficient sequential selection of the population with the largest mean subject to a correct selection probability constraint. The selection procedure consists of a sampling rule, a stopping rule, and a terminal decision rule. Efficiency at every parameter configuration is measured by the expected total sampling cost together with the correct selection probability. By using sequential generalized likelihood ratio tests of multiple hypotheses and an adaptive sampling rule based on a constrained optimization problem, we show that it is possible to achieve asymptotic efficiency at the true (but unknown) parameter configuration as the probability of incorrect selection approaches 0, thereby resolving a number of open problems in this area. Finite-sample efficiency of the proposed procedure is demonstrated in simulation studies that also compare the procedure with other sequential selection procedures in the literatur...

T. L. Lai

Papers

Sequential generalized likelihood ratios and adaptive treatment allocation for optimal sequential selection