Statistica Sinica 10(2000), 1-50
HYBRID RESAMPLING METHODS FOR CONFIDENCE
INTERVALS
Chin-Shan Chuang and Tze Leung Lai
Carnegie-Mellon University and Stanford University
Abstract: This paper considers t he problem of constructing confidence intervals
for a single parameter θ in a multiparameter or nonparametric family. Hybrid
resampling methods, which “hybridize” the essential features of bootstrap and ex-
act methods, are proposed and developed for both parametric and nonparametric
situations. In particular, we apply such methods to construct confidence regions,
whose coverage probabilities are nearly equal to the nominal ones, for the treat-
ment effects associated with the primary and secondary endpoints of a clinical trial
whose stopping rule, specified by a group sequential test, makes the approximate
pivots in the nonsequential b ootstrap method highly “non-pivotal”. We also apply
hybrid resampling methods to construct second-order correct confidence intervals
in possibly non-ergod ic autoregressive models and branching processes.
Key words and phrases: Bootstrap confidence intervals, empirical likelihood, group
sequential tests, h ybrid resampling, nonparametric tilting.
1. Introduction
The past two decades have witnessed important developments in group se-
quential methods for interim analysis of clinical trials. Although these methods
allow for early termination while preserving the overall significance level of the
test, they introduce substantial difficulties in constructing confidence intervals for
parameters of interest following the test. Under strong distributional assump-
tions and for r elatively simple parametric mo dels, exact confidence intervals in
group sequential settings have b een developed in the literature. For samples of
fixed size, an important methodology for constructing confidence intervals with-
out distributional assumptions is the bootstrap metho d. Although the stopping
rule makes approximate pivots in the nonsequential bo otstrap metho d highly
“non-piv otal”, we have recently shown in Chuang and Lai (1998) that it is possi-
ble to “hybridize” the bootstrap and exact methods for constructing confidence
in tervals follo wing a group sequential test.
In Sections 2, 3 and 6, we give a comprehensive development of the hybrid
resampling approach, extending the metho dology beyond the group sequential
setting considered in Chuang and Lai (1998) and in Section 4 of the present paper.
2 CHIN-SHAN CHUANG AND TZE LEUNG LAI
In particular, in Section 5, we show how the metho dology can b e used to con-
struct second-order correct confidence intervals for the autoregressive parameter
θ of a possibly nonstationary AR(1) model, for which the bootstrap method has
been shown to b e inconsistent when |θ| =1. Since the bo otstrap method is a spe-
cial case of hybrid resampling as shown in Section 2, our development of hybrid
resampling also addresses certain basic issues concerning the bootstrap method,
such as choice of root, difficulties with the infinitesimal jackknife and lineariza-
tion, influential observations, calibration and bootstrap iteration. Moreover, in
connection with the choice of a resampling family for the hybrid approach, we
also discuss certain basic issues concerning Owen’s (1988, 1990) empirical likeli-
hood and Efron’s (1981, 1987) nonparametric tilting. Some concluding remarks
are given in Section 7.
2. Exact, Bootstrap and Hybrid Resampling Methods for Confidence
Intervals
We b egin with the use of exact, bo otstrap and hybrid resampling methods
for constructing confidence intervals. Let X =(X
11
,...,X
1p
,...,X
n1
,...,X
np
)
be a vector of observations from some family of distributions {F : F ∈F}. For
nonparametric problems, F is the family of distributions on R
p
satisfying certain
prespecified regularity conditions and (X
i1
,...,X
ip
) are i.i.d. rrandom v ectors
having common distribution F ∈F. For parametric models with parameter η ∈
Γ, we can denote F by {F
η
: η ∈ Γ },and(X
i1
,...,X
ip
) may be i.i.d. or may
form a time series. The problem of interest is to find a confidence interval for the
real-valued parameter θ = θ(F ). We let Θ denote the set of all possible values of
θ.
Exact method:IfF = {F
θ
: θ ∈ Θ} is indexed by a real-valued parameter
θ, an exact equal-tailed confidence region can always be found by using the
well known duality between hypothesis tests and confidence regions (cf. Rosner
and Tsiatis (1988), Schenk er (1987)). Suppose one would like to test the null
h ypothesis that θ is equal to θ
0
. Let R(X,θ
0
) be some real-valued test statistic.
Let u
α
(θ
0
)betheα-quantile of the distribution of R(X,θ
0
) under the distribution
F
θ
0
. The null hypothesis is accepted if u
α
(θ
0
) <R(X,θ
0
) <u
1−α
(θ
0
). An exact
equal-tailed confidence region with coverage probability 1 −2α consists of all θ
0
not rejected by the test and is therefore given by
{θ : u
α
(θ) <R(X,θ) <u
1−α
(θ)}. (2.1)
Bootstrap method: The exact method applies only when there are no nuisance
parameters and this assumption is rarely satisfied in practice. The bootstrap
method replaces the quantiles u
α
(θ)andu
1−α
(θ) by the approximate quantiles
HYBRID RESAMPLING METHODS 3
u
∗
α
and u
∗
1−α
obtained in the following manner. Based on X, construct an estimate
F of F ∈F. The quantile u
∗
α
is defined to be the α-quantile of the distribution
of R(X
∗
,
θ)withX
∗
generated from
F and
θ = θ(
F ); see Efron (1981, 1987).
Thus, the bootstrap method yields the following confidence region for θ with
approximate co verage probability 1 − 2α:
{θ : u
∗
α
<R(X,θ) <u
∗
1−α
}. (2.2)
In particular, when
F is the empirical distribution of i.i.d. X
1
,...,X
n
and the
ro ot R(X,θ)isequalto(
θ − θ)/
σ for some estimate
σ of the standard error of
θ, the bootstrap confidence interval (2.2) is called the bo otstrap-t interval. It
is well known that the bootstrap-t interval has coverage error O(n
−1
)atboth
endp oints when θ is a smooth function of means; see Hall (1988, 1992).
Hybrid resampling method: The hybrid confidence region is based on reduc-
ing the family of distributions F to another family of distributions {
F
θ
: θ ∈ Θ},
where θ is the unknown parameter of interest. We call this family the “resam-
pling family”. This reduction depends on X, and ways for carrying it out are
explored in the rest of the paper. Let
u
α
(θ)betheα-quantile of the sampling
distribution of R(X,θ) under the assumption that X has distribution
F
θ
. The
hybrid confidence region results from applying the exact method to {
F
θ
: θ ∈ Θ}
and is giv en by
{θ :
u
α
(θ) <R(X,θ) <
u
1−α
(θ)}. (2.3)
The construction of (2.3) typically involves simulations to compute the quantiles
as in the bootstrap method and is elaborated below. We call this the “hybrid
resampling” method because it “hybridizes” the exact method (that uses test
inversion) with the bootstrap method (that uses the observe d data to determine
the resampling distribution). Note that hybrid resampling is a generalization of
the bootstrap method, which uses the singleton {
F } as the resampling family
{
F
θ
}. The following two examples, which will be discussed in Sections 4 and 5,
illustrate difficulties with the bootstrap method when the sampling distribution
of R(X,θ) may vary substantially with θ.
Example 1. Consider a group sequential trial with Pocock’s (1977) boundary
and a maximum of 5 groups, as in Chuang and Lai (1998). Let X
1
,X
2
,... be
independent with mean θ and variance 1, S
n
= X
1
+ ···+ X
n
,
¯
X
n
= S
n
/n,and
let J = {15j : j =1, 2, 3, 4, 5}. The stopping rule is τ =min{n ∈ J : |S
n
|≥
2.413n
1/2
}, where we define the minimum of ∅ to be 75 = 15 × 5. The choice
2.413 ensures that when θ =0,P{max
n∈J
|S
n
/n
1/2
|≥2.413}
.
=0.05; see Pocock
(1977). Figure 1 in Chuang and Lai (1998) shows that the sampling distribution
of
√
τ(
¯
X
τ
−θ) varies markedly with θ even for normal observations, and Table 1
there reports poor performance of the bootstrap method.
4 CHIN-SHAN CHUANG AND TZE LEUNG LAI
Example 2. Consider the first-order autoregressive AR(1) model given by the
following. Let x
0
=0andx
i
= θx
i−1
+ ǫ
i
, where ǫ
i
are i.i.d. with mean 0 and
variance 1. Let
θ be the least squares estimate of θ based on (x
1
,...,x
n
), given
by
θ =
n
i=1
x
i
x
i−1
/
n
i=1
x
2
i−1
. It is well known that the limiting distribution
of (
n
i=1
x
2
i−1
)
1/2
(
θ − θ) is standard normal if |θ| < 1. However, when |θ| =1,
the limiting distribution is given by
1
2
(B
2
1
− 1)/(
1
0
B
2
t
dt)
1/2
, where B
t
denotes
standard Brownian motion. Basawa, Mallik, McCormick, Reev es and Taylor
(1991) showed that bootstrapping the least squares estimate is inconsistent when
|θ| =1.
In practice, it is often desirable to express a confidence set for θ as an interval.
Although the sets (2.1), (2.2) and (2.3) may not be intervals, it often suffices to
give only the upper and lower limits of the confidence set. We now describ e
an algorithm, based on the method of successive secan t approximations, to find
the upper limit of (2.3). Let f(θ)=R(X,θ) −
u
α
(θ) and consider solving the
equation f (θ)=0. First we find a
1
<b
1
such that f(a
1
) > 0andf(b
1
) < 0. Let
f
1
(θ) be linear in θ ∈ [a
1
,b
1
]withf
1
(a
1
)=f(a
1
)andf
1
(b
1
)=f(b
1
), and let θ
1
be the root of f
1
(θ)=0. If f (θ
1
) > 0, set a
2
= θ
1
and b
2
= b
1
. If f (θ
1
) < 0, set
b
2
= θ
1
and a
2
= a
1
. Proceeding inductively in this manner, we let f
k
(θ) linearly
in terpolate f(a
k
)andf(b
k
)fora
k
≤ θ ≤ b
k
, and let θ
k
∈ (a
k
,b
k
) be the root of
f
k
(θ)=0. This procedure terminates if θ
k
differs little from θ
k−1
or if k reaches
some upper bound, and the terminal value θ
k
is taken to be the upper limit of
(2.3). Typically f(
θ) > 0, so
θ canbechosenasa
1
. To find b
1
, one can start with
b
′
1
=
θ +2
σ, where
σ is an estimate of the standard error of
θ. If f(b
′
1
) < 0, set
b
1
= b
′
1
;otherwiseletb
′
2
= b
′
1
+
σ/2 and check whether f(b
′
2
) < 0. This procedure
is repeated until one arrives at f(b
′
h
) < 0andsetsb
1
= b
′
h
. The total n umber of
iterations, h + k, is kept no more than some prescribed upper bound m. If there
are already m iterations before one arrives at b
1
with f(b
1
) < 0(soh = m),
take b
′
m
as the default value of the upper limit of (2.3). For the simulations in
Example 3 and those used to produce Tables 4 and 5, we took m =8. For the
simulations in Example 7 of Section 6, we took m = 4 to ease the computational
burden. The quantiles
u
α
(θ
j
) can be computed from indep endent samples from
F
θ
j
, as was done in these examples. It is sometimes possible to try to reuse
the same random sample for all θ values, as in the mean and regression models
considered in Tables 4 and 5, but there is not much computational saving since
we typically do not use a large value of m.
3. Choice of Root and Implementation of Hybrid Resampling Methods
As the framework of Section 2 suggests, there are two issues that must b e
addressed for hybrid resampling methods to be used successfully in practice.
HYBRID RESAMPLING METHODS 5
First, one m ust choose an appropriate root R(X,θ) that is used in resampling.
Second, one needs to find a suitable reduction of the original family F to the
resampling family {
F
θ
}. The second issue for nonparametric problems is quite
complicated and is deferred to Section 6; in the examples considered in Section
4 and 5, there is a simple reduction.
3.1. Choice of root and resampling family in parametric models
One natural root R(X,θ) that can be used in parametric models is the signed
root of the log likelihood ratio statistic. Let X represent a vector of observations
from a parametric family F
η
with joint density f(x; η), and let θ = g (η)be
a real-valued parameter of interest. Consider testing the null hypothesis that
θ = θ
0
. Let
η be the (unrestricted) maximum likelihood estimate of η based on
X and let
η(θ
0
) be the maximum likelihood estimate of η subject to the constraint
g(η)=θ
0
. The likelihood ratio test rejects the null hypothesis for large values of
l(θ
0
)=2{log f(X;
η) − log f(X;
η(θ
0
))}. (3.1)
Equivalently, one rejects the null hypothesis for large absolute values of the signed
ro ot
l
±
(θ
0
)=sgn(
θ − θ
0
)l
1/2
(θ
0
). (3.2)
Here,
θ represents the maximum likelihood estimate of θ based on X, i.e.
θ =
g(
η). With R(X,θ)=l
±
(θ), the quantiles u
∗
α
and u
∗
1−α
used in the bootstrap
confidence region (2.2) are determined from the distribution of l
±
(θ) under the
assumption that X is generated from F
η
. The quantiles
u
α
(θ)and
u
1−α
(θ)usedin
the hybrid confidence region (2.3) are determined from the distribution of l
±
(θ)
under the assumption that X is generated from F
η(θ)
. Note that the hybrid
region (2.3) is obtained by reducing the family F
η
to the family F
η(θ)
, whereas
the bootstrap confidence region (2.2) is based on the single distribution F
η
.
Example 3. Consider the following Galton-Watson branching process with im-
migration (BPI). Let ξ
1
,ξ
2
,... be i.i.d. Poisson random variables with mean θ,
and let ψ
1
,ψ
2
,...be i.i.d. Pois son random variables with mean λ. Here, the mean
θ of the offspring distribution is of inte rest, whereas λ is regarded as a nuisance
parameter. The Galton-Watson BPI is defined as follows. Assume that X
0
= x
0
for some positive integer x
0
. Let X
1
= ξ
1
+ ···+ ξ
x
0
+ ψ
1
, which represents the
size of the first generation. The size X
n
of the nth generation, for n =2, 3,...,
is given by
X
n
= ξ
x
0
+X
1
+···+X
n−2
+1
+ ···+ ξ
x
0
+X
1
+···+X
n−1
+ ψ
n
.