scispace - formally typeset
Search or ask a question
Journal ArticleDOI

A Bernoulli Two-armed Bandit

01 Jun 1972-Annals of Mathematical Statistics (Institute of Mathematical Statistics)-Vol. 43, Iss: 3, pp 871-897
TL;DR: In this article, a Bernoulli process with unknown expectations is selected and observed at each of n$ stages, and the objective is to maximize the expected number of successes from the n$ selections.
Abstract: One of two independent Bernoulli processes (arms) with unknown expectations $\rho$ and $\lambda$ is selected and observed at each of $n$ stages. The selection problem is sequential in that the process which is selected at a particular stage is a function of the results of previous selections as well as of prior information about $\rho$ and $\lambda$. The variables $\rho$ and $\lambda$ are assumed to be independent under the (prior) probability distribution. The objective is to maximize the expected number of successes from the $n$ selections. Sufficient conditions for the optimality of selecting one or the other of the arms are given and illustrated for example distributions. The stay-on-a-winner rule is proved.
Citations
More filters
Journal ArticleDOI
Karl Claxton1
TL;DR: It is argued here that rules of inference are arbitrary and entirely irrelevant to the decisions which clinical and economic evaluations claim to inform and a framework for decision making and establishing the value of additional information is presented which is consistent with the decision rules in CEA.

873 citations

01 Jan 2001
TL;DR: A brief review of the developments in several classical problems of sequential analysis and their applications to biomedicine, economics and engi- neering can be found in this paper.
Abstract: We give a brief review of the developments in several classical problems of sequential analysis and their applications to biomedicine, economics and engi- neering. Even though it can only focus on a limited number of topics, the review shows that sequential analysis is still a vibrant subject after six decades of contin- ual development, with fresh ideas brought in from various fields of application and through interactions with other branches of statistics and probability. We conclude with some remarks on the opportunities and challenges ahead.

423 citations


Cites background from "A Bernoulli Two-armed Bandit"

  • ...In the case of k = 2 Bernoulli populations with independent Beta priors for their parameters, Fabius and van Zwet (1970) and Berry (1972) studied the dynamic programming equations analytically and obtained several qualitative results concerning the optimal rule....

    [...]