Showing papers by "Donald B. Rubin published in 2003"

r equivalent: A simple effect size indicator.

Abstract: The purpose of this article is to propose a simple effect size estimate (obtained from the sample size, N, and a p value) that can be used (a) in meta-analytic research where only sample sizes and p values have been reported by the original investigator, (b) where no generally accepted effect size estimate exists, or (c) where directly computed effect size estimates are likely to be misleading. This effect size estimate is called r(equivalent) because it equals the sample point-biserial correlation between the treatment indicator and an exactly normally distributed outcome in a two-treatment experiment with N/2 units in each group and the obtained p value. As part of placing r(equivalent) into a broader context, the authors also address limitations of r(equivalent).

Estimation of Causal Effects via Principal Stratification When Some Outcomes are Truncated by “Death”

Abstract: The topic of “truncation by death” in randomized experiments arises in many fields, such as medicine, economics and education. Traditional approaches addressing this issue ignore the fact that the outcome after the truncation is neither “censored” nor “missing,” but should be treated as being defined on an extended sample space. Using an educational example to illustrate, we will outline here a formulation for tackling this issue, where we call the outcome “truncated by death” because there is no hidden value of the outcome variable masked by the truncating event. We first formulate the principal stratification (Frangakis & Rubin, 2002) approach, and we then derive large sample bounds for causal effects within the principal strata, with or without various identification assumptions. Extensions are then briefly discussed.

Comment: Principal stratification approach to broken randomized experiments: A case study of school choice vouchers in New York City

Abstract: The precarious state of the educational system in the inner cities of the United States, as well as its potential causes and solutions, have been popular topics of debate in recent years. Part of the difficulty in resolving this debate is the lack of solid empirical evidence regarding the true impact of educational initiatives. The efficacy of so-called “school choice” programs has been a particularly contentious issue. A current multimillion dollar program, the School Choice Scholarship Foundation Program in New York, randomized the distribution of vouchers in an attempt to shed some light on this issue. This is an important time for school choice, because on June 27, 2002 the U.S. Supreme Court upheld the constitutionality of a voucher program in Cleveland that provides scholarships both to secular and religious private schools. Although this study benefits immensely from a randomized design, it suffers from complications common to such research with human subjects: noncompliance with assigned “treatmen...

Nested multiple imputation of NMES via partially incompatible MCMC

Abstract: The multiple imputation of the National Medical Expenditure Survey (NMES) involved the use of two new techniques, both having potentially broad applicability. The first is to use distributionally incompatible MCMC (Markov Chain Monte Carlo), but to apply it only partially, to impute the missing values that destroy a monotone pattern, thereby limiting the extent of incompatibility. The second technique is to split the missing data into two parts, one that is much more computationally expensive to impute than the other, and create several imputations of the second part for each of the first part, thereby creating nested multiple imputations with their increased inferential efficiency.

Principal stratification approach to broken randomized experiments: A case study of school choice vouchers in New York City. Comments. Authors' reply

Abstract: The precarious state of the educational system in the inner cities of the United States, as well as its potential causes and solutions, have been popular topics of debate in recent years. Part of the difficulty in resolving this debate is the lack of solid empirical evidence regarding the true impact of educational initiatives The efficacy of so-called school choice programs has been a particularly contentious issue. A current multimillion dollar program, the School Choice Scholarship Foundation Program in New York, randomized the distribution of vouchers in an attempt to shed some light on this issue. This is an important time for school choice, because on June 27, 2002 the U.S. Supreme Court upheld the constitutionality of a voucher program in Cleveland that provides scholarships both to secular and religious private schools. Although this study benefits immensely from a randomized design, it suffers from complications common to such research with human subjects: noncompliance with assigned treatments and missing data. Recent work has revealed threats to valid estimates of experimental effects that exist in the presence of noncompliance and missing data, even when the goal is to estimate simple intention-to-treat effects. Our goal was to create a better solution when faced with both noncompliance and missing data. This article presents a model that accommodates these complications that is based on the general framework of principal stratification and thus relies on more plausible assumptions than standard methodology. Our analyses revealed positive effects on math scores for children who applied to the program from certain types of schools-those with average test scores below the citywide median. Among these children, the effects are stronger for children who applied in the first grade and for African-American children.

Hypothesis: A single clinical trial plus causal evidence of effectiveness is sufficient for drug approval

Abstract: In 1997, President Clinton signed the Food and Drug Administration Modernization Act of 1997 (FDAMA). Among its many provisions, section 115a amended the Federal Food, Drug, and Cosmetics Act to permit determination of substantial evidence of effectiveness as required for approval of a new drug to be based on “data from one adequate and well-controlled investigation and confirmatory evidence.”* This language contrasts to the statute’s previous wording, introduced in the 1962 amendment, that required “adequate and well controlled investigation s” (note plural, emphasis added) and interpreted by the US Food and Drug Administration to require (at least) 2 such trials. Exactly how the new mandate of FDAMA 1997 should be interpreted has since been a matter of as yet unresolved debate (see, for example, Peck and Wechsler ). The purposes of this commentary are to review the historical basis for the new amendment, present a logical framework for it, and propose a direction its implementation could take. We do this not so much to propose definitive policy as to stimulate discussion on what we believe to be one of the most important issues in drug regulation, raised anew by section 115a, namely, the evidentiary standards appropriate for concluding efficacy. We argue that the only reason a first successful randomized clinical trial (RCT) must be replicated under the 2-trial paradigm is to rule out nonreplicable causes of the success of the first RCT and thereby From the Center for Drug Development Science, Office of the As sociate Dean for Clinical Research, Georgetown University School of Medicine, Washington; Department of Statistics, Harvard Uni versity, Cambridge; and Departments of Laboratory Medicine and Biopharmaceutical Sciences, Schools of Medicine and Pharmacy, University of California, San Francisco. Received for publication Dec 27, 2001; accepted Jan 6, 2003. Reprint requests: Lewis B. Sheiner, Box 0626, UCSF, San Francisco, CA 94143-0626. *Specifically, paragraph 505(d) of the Food, Drug, and Cosmetic act was modified (in italics) as follows: “The term ’substantial evidence’ means evidence consisting of adequate and well-controlled investigations, including clinical investigations...on the basis of which it could fairly and responsibly be concluded...that the drug will have the effect it purports or is represented to have... if the secretary determines, based on relevant science, that data from one adequate and well-controlled clinical investigation and confirmatory evidence (obtained prior to or after such investigation) are sufficient to establish effectiveness, the secretary may consider such data and evidence to constitute substantial evidence. . . .” Clin Pharmacol Ther 2003;73:481-90. Copyright © 2003 by the American Society for Clinical Pharmacology & Therapeutics. 0009-9236/2003/$30.00 0 doi:10.1016/S0009-9236(03)00018-3

Finding the "genuine" schizotype: a model and method for resolving heterogeneity in performance on laboratory measures in experimental psychopathology research.

Abstract: Heterogeneity in the performance of persons affected with schizophrenia or schizotypic psychopathology on various laboratory tasks has long been recognized, both for its consistency across tasks and studies and for the massive methodological and substantive challenges it poses for experimental psychopathology, genetic, and other investigations. Traditional multivariate techniques, such as factor analysis, discriminant function analysis, and cluster analysis, have all been deemed inadequate for resolving heterogeneity, because of one or another statistical limitation. Here, an objective statistical approach based on a formal statistical model that uses the ubiquitous and well-developed expectation-maximization (EM) algorithm (A. P. Dempster, N. M. Laird, & D. B. Rubin, 1977) is presented, which enables one effectively to partition a group of experimental subjects, in this case identified initially using the well-known Perceptual Aberration Scale (L. J. Chapman, J. P. Chapman, & M. L. Raulin, 1978), in a manner that reduces heterogeneity and allows for the separation of what are termed genuine and false-positive schizotypes. The validity of the parsing strategy was supported by reference to other laboratory indexes of relevance to schizophrenia and schizotypy that were not included in the initial EM-based analyses. The potential utility of this approach is discussed with reference to future schizophrenia and schizotypy research.

Taking causality seriously: propensity score methodology applied to estimate the effects of marketing interventions

Abstract: Propensity score methods were proposed by Rosenbaum and Rubin (1983, Biometrika) as central tools to help assess the causal effects of interventions Since their introduction two decades ago, they have found wide application in a variety of areas, including medical research, economics, epidemiology, and education, especially in those situations where randomized experiments are either difficult to perform, or raise ethical questions, or would require extensive delays before answers could be obtained Rubin (1997, Annals of Internal Medicine) provides an introduction to some of the essential ideas In the past few years, the number of published applications using propensity score methods to evaluate medical and epidemiological interventions has increased dramatically Rubin (2003, Erlbaum) provides a summary, which is already out of date