Pairwise Randomized Experiments

TLDR: In this paper, the Neyman sampling variance estimator was used to analyze the within-stratum estimates in pairwise randomized experiments, where each stratum contains exactly two units, with one randomly selected to be assigned to the treatment group, and the other one assigned to control group.

Abstract: INTRODUCTION In the previous chapter we analyzed stratified randomized experiments, where a sample of size N was partitioned into J strata, and within each stratum a completely randomized experiment was conducted. In this chapter we consider a special case of the stratified randomized experiment. Each stratum contains exactly two units, with one randomly selected to be assigned to the treatment group, and the other one assigned to the control group. Such a design is known as a pairwise randomized experiment or paired comparison . Although this can be viewed simply as a special case of a stratified randomized experiment, there are two features of this design that warrant special attention. First, the fact that there is only a single unit in each treatment group in each stratum (or pair in this case) implies that the Neyman sampling variance estimator that we discussed in the chapters on completely randomized experiments (Chapter 6) and stratified randomized experiments (Chapter 9) cannot be used; that estimator requires the presence of at least two units assigned to each treatment in each stratum. Second, each stratum has the same proportion of treated units, which allows us to analyze the within-stratum estimates symmetrically; the natural estimator for the average treatment effect weights each stratum equally. As in the case of stratified randomized experiments, the motivation for eliminating some of the possible assignments in pairwise randomized experiments is that a priori those values of the assignment vectors that are eliminated are expected to lead to less informative inferences. This argument relies on the within-pair variation in potential outcomes being small relative to the between-pair variation. Often the assignment to pairs is based on covariates. Units are matched to other units based on their similarity in these covariates, with the expectation that this similarity corresponds to similarity in the potential outcomes under each treatment. Suppose, for example, that the treatment is an expensive surgical procedure for a relatively common medical condition. It may not be financially feasible to apply the treatment to many individuals.