THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

Empirical software engineering research needs research guidelines to improve the research and reporting processes. We propose a preliminary set of research guidelines aimed at stimulating discussion among software researchers. They are based on a review of research guidelines developed for medical researchers and on our own experience in doing and reviewing software engineering research. The guidelines are intended to assist researchers, reviewers, and meta-analysts in designing, conducting, and evaluating empirical studies. Editorial boards of software engineering journals may wish to use our recommendations as a basis for developing guidelines for reviewers and for framing policies for dealing with the design, data collection, and analysis and reporting of empirical studies.

/pdf/preliminary-guidelines-for-empirical-research-in-software-17mchb417s.pdf

Preliminary guidelines for empirical research in software engineering

There is an explosion of interest in Bayesian statistics, primarily because recently created computational methods have finally made Bayesian analysis obtainable to a wide audience. Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan provides an accessible approach to Bayesian data analysis, as material is explained clearly with concrete examples. The book begins with the basics, including essential concepts of probability and random sampling, and gradually progresses to advanced hierarchical modeling methods for realistic data. Included are step-by-step instructions on how to conduct Bayesian data analyses in the popular and free software R and WinBugs. This book is intended for first-year graduate students or advanced undergraduates. It provides a bridge between undergraduate training and modern Bayesian methods for data analysis, which is becoming the accepted research standard. Knowledge of algebra and basic calculus is a prerequisite. New to this Edition (partial list): * There are all new programs in JAGS and Stan. The new programs are designed to be much easier to use than the scripts in the first edition. In particular, there are now compact high-level scripts that make it easy to run the programs on your own data sets. This new programming was a major undertaking by itself.* The introductory Chapter 2, regarding the basic ideas of how Bayesian inference re-allocates credibility across possibilities, is completely rewritten and greatly expanded.* There are completely new chapters on the programming languages R (Ch. 3), JAGS (Ch. 8), and Stan (Ch. 14). The lengthy new chapter on R includes explanations of data files and structures such as lists and data frames, along with several utility functions. (It also has a new poem that I am particularly pleased with.) The new chapter on JAGS includes explanation of the RunJAGS package which executes JAGS on parallel computer cores. The new chapter on Stan provides a novel explanation of the concepts of Hamiltonian Monte Carlo. The chapter on Stan also explains conceptual differences in program flow between it and JAGS.* Chapter 5 on Bayes' rule is greatly revised, with a new emphasis on how Bayes' rule re-allocates credibility across parameter values from prior to posterior. The material on model comparison has been removed from all the early chapters and integrated into a compact presentation in Chapter 10.* What were two separate chapters on the Metropolis algorithm and Gibbs sampling have been consolidated into a single chapter on MCMC methods (as Chapter 7). There is extensive new material on MCMC convergence diagnostics in Chapters 7 and 8. There are explanations of autocorrelation and effective sample size. There is also exploration of the stability of the estimates of the HDI limits. New computer programs display the diagnostics, as well.* Chapter 9 on hierarchical models includes extensive new and unique material on the crucial concept of shrinkage, along with new examples.* All the material on model comparison, which was spread across various chapters in the first edition, in now consolidated into a single focused chapter (Ch. 10) that emphasizes its conceptualization as a case of hierarchical modeling.* Chapter 11 on null hypothesis significance testing is extensively revised. It has new material for introducing the concept of sampling distribution. It has new illustrations of sampling distributions for various stopping rules, and for multiple tests.* Chapter 12, regarding Bayesian approaches to null value assessment, has new material about the region of practical equivalence (ROPE), new examples of accepting the null value by Bayes factors, and new explanation of the Bayes factor in terms of the Savage-Dickey method.* Chapter 13, regarding statistical power and sample size, has an extensive new section on sequential testing, and making the research goal be precision of estimation instead of rejecting or accepting a particular value.* Chapter 15, which introduces the generalized linear model, is fully revised, with more complete tables showing combinations of predicted and predictor variable types.* Chapter 16, regarding estimation of means, now includes extensive discussion of comparing two groups, along with explicit estimates of effect size.* Chapter 17, regarding regression on a single metric predictor, now includes extensive examples of robust regression in JAGS and Stan. New examples of hierarchical regression, including quadratic trend, graphically illustrate shrinkage in estimates of individual slopes and curvatures. The use of weighted data is also illustrated.* Chapter 18, on multiple linear regression, includes a new section on Bayesian variable selection, in which various candidate predictors are probabilistically included in the regression model.* Chapter 19, on one-factor ANOVA-like analysis, has all new examples, including a completely worked out example analogous to analysis of covariance (ANCOVA), and a new example involving heterogeneous variances.* Chapter 20, on multi-factor ANOVA-like analysis, has all new examples, including a completely worked out example of a split-plot design that involves a combination of a within-subjects factor and a between-subjects factor.* Chapter 21, on logistic regression, is expanded to include examples of robust logistic regression, and examples with nominal predictors.* There is a completely new chapter (Ch. 22) on multinomial logistic regression. This chapter fills in a case of the generalized linear model (namely, a nominal predicted variable) that was missing from the first edition.* Chapter 23, regarding ordinal data, is greatly expanded. New examples illustrate single-group and two-group analyses, and demonstrate how interpretations differ from treating ordinal data as if they were metric.* There is a new section (25.4) that explains how to model censored data in JAGS.* Many exercises are new or revised. * Accessible, including the basics of essential concepts of probability and random sampling* Examples with R programming language and JAGS software* Comprehensive coverage of all scenarios addressed by non-Bayesian textbooks: t-tests, analysis of variance (ANOVA) and comparisons in ANOVA, multiple regression, and chi-square (contingency table analysis)* Coverage of experiment planning* R and JAGS computer programming code on website* Exercises have explicit purposes and guidelines for accomplishment* Provides step-by-step instructions on how to conduct Bayesian data analyses in the popular and free software R and WinBugs

Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan

Consider a study whose design calls for the study subjects to be followed from enrollment (time t = 0) to time t = T, at which point a primary endpoint of interest Y is to be measured. The design of the study also calls for measurements on a vector V t) of covariates to be made at one or more times t during the interval [0, T). We are interested in making inferences about the marginal mean μ0 of Y when some subjects drop out of the study at random times Q prior to the common fixed end of follow-up time T. The purpose of this article is to show how to make inferences about μ0 when the continuous drop-out time Q is modeled semiparametrically and no restrictions are placed on the joint distribution of the outcome and other measured variables. In particular, we consider two models for the conditional hazard of drop-out given (V(T), Y), where V(t) denotes the history of the process V t) through time t, t ∈ [0, T). In the first model, we assume that λQ(t|V(T), Y) exp(α0 Y), where α0 is a scalar paramet...

/pdf/adjusting-for-nonignorable-drop-out-using-semiparametric-3qq6f67r97.pdf

Adjusting for Nonignorable Drop-Out Using Semiparametric Nonresponse Models

Preface. List of examples. 1. Introduction. 1.1 What are Bayesian methods? 1.2 What do we mean by 'health--care evaluation'? 1.3 A Bayesian approach to evaluation. 1.4 The aim of this book and the intended audience. 1.5 Structure of the book. 2. Basic Concepts from Traditional Statistical Analysis. 2.1 Probability. 2.1.1 What is probability? 2.1.2 Odds and log--odds. 2.1.3 Bayes theorem for simple events. 2.2 Random variables, parameters and likelihood. 2.2.1 Random variables and their distributions. 2.2.2 Expectation, variance, covariance and correlation. 2.2.3 Parametric distributions and conditional independence. 2.2.4 Likelihoods. 2.3 The normal distribution. 2.4 Normal likelihoods. 2.4.1 Normal approximations for binary data. 2.4.2 Normal likelihoods for survival data. 2.4.3 Normal likelihoods for count responses. 2.4.4 Normal likelihoods for continuous responses. 2.5 Classical inference. 2.6 A catalogue of useful distributionsaeo. 2.6.1 Binomial and Bernoulli. 2.6.2 Poisson. 2.6.3 Beta. 2.6.4 Uniform. 2.6.5 Gamma. 2.6.6 Root--inverse--gamma. 2.6.7 Half--normal. 2.6.8 Log--normal. 2.6.9 Student's t. 2.6.10 Bivariate normal. 2.7 Key points. Exercises. 3. An Overview of the Bayesian Approach. 3.1 Subjectivity and context. 3.2 Bayes theorem for two hypotheses. 3.3 Comparing simple hypotheses: likelihood ratios and Bayes factors. 3.4 Exchangeability and parametric modellingaeo. 3.5 Bayes theorem for general quantities. 3.6 Bayesian analysis with binary data. 3.6.1 Binary data with a discrete prior distribution. 3.6.2 Conjugate analysis for binary data. 3.7 Bayesian analysis with normal distributions. 3.8 Point estimation, interval estimation and interval hypotheses. 3.9 The prior distribution. 3.10 How to use Bayes theorem to interpret trial results. 3.11 The 'credibility' of significant trial resultsaeo. 3.12 Sequential use of Bayes theoremaeo. 3.13 Predictions. 3.13.1 Predictions in the Bayesian framework. 3.13.2 Predictions for binary dataaeo. 3.13.3 Predictions for normal data. 3.14 Decision--making. 3.15 Design. 3.16 Use of historical data. 3.17 Multiplicity, exchangeability and hierarchical models. 3.18 Dealing with nuisance parametersaeo. 3.18.1 Alternative methods for eliminating nuisance parametersaeo. 3.18.2 Profile likelihood in a hierarchical modelaeo. 3.19 Computational issues. 3.19.1 Monte Carlo methods. 3.19.2 Markov chain Monte Carlo methods. 3.19.3 WinBUGS. 3.20 Schools of Bayesians. 3.21 A Bayesian checklist. 3.22 Further reading. 3.23 Key points. Exercises. 4. Comparison of Alternative Approaches to Inference. 4.1 A structure for alternative approaches. 4.2 Conventional statistical methods used in health--care evaluation. 4.3 The likelihood principle, sequential analysis and types of error. 4.3.1 The likelihood principle. 4.3.2 Sequential analysis. 4.3.3 Type I and Type II error. 4.4 P--values and Bayes factorsaeo. 4.4.1 Criticism of P--values. 4.4.2 Bayes factors as an alternative to P--values: simple hypotheses. 4.4.3 Bayes factors as an alternative to P--values: composite hypotheses. 4.4.4 Bayes factors in preference studies. 4.4.5 Lindley's paradox. 4.5 Key points. Exercises. 5. Prior Distributions. 5.1 Introduction. 5.2 Elicitation of opinion: a brief review. 5.2.1 Background to elicitation. 5.2.2 Elicitation techniques. 5.2.3 Elicitation from multiple experts. 5.3 Critique of prior elicitation. 5.4 Summary of external evidenceaeo. 5.5 Default priors. 5.5.1 'Non--informative' or 'reference' priors: 5.5.2 'Sceptical' priors. 5.5.3 'Enthusiastic' priors. 5.5.4 Priors with a point mass at the null hypothesis ('lump--and--smear' priors)aeo. 5.6 Sensitivity analysis and 'robust' priors. 5.7 Hierarchical priors. 5.7.1 The judgement of exchangeability. 5.7.2 The form for the random--effects distribution. 5.7.3 The prior for the standard deviation of the random effectsaeo. 5.8 Empirical criticism of priors. 5.9 Key points. Exercises. 6. Randomised Controlled Trials. 6.1 Introduction. 6.2 Use of a loss function: is a clinical trial for inference or decision? 6.3 Specification of null hypotheses. 6.4 Ethics and randomisation: a brief review. 6.4.1 Is randomisation necessary? 6.4.2 When is it ethical to randomise? 6.5 Sample size of non--sequential trials. 6.5.1 Alternative approaches to sample--size assessment. 6.5.2 'Classical power': hybrid classical--Bayesian methods assuming normality. 6.5.3 'Bayesian power'. 6.5.4 Adjusting formulae for different hypotheses. 6.5.5 Predictive distribution of power and necessary sample size. 6.6 Monitoring of sequential trials. 6.6.1 Introduction. 6.6.2 Monitoring using the posterior distribution. 6.6.3 Monitoring using predictions: 'interim power'. 6.6.4 Monitoring using a formal loss function. 6.6.5 Frequentist properties of sequential Bayesian methods. 6.6.6 Bayesian methods and data monitoring committees. 6.7 The role of 'scepticism' in confirmatory studies. 6.8 Multiplicity in randomised trials. 6.8.1 Subset analysis. 6.8.2 Multi--centre analysis. 6.8.3 Cluster randomization. 6.8.4 Multiple endpoints and treatments. 6.9 Using historical controlsaeo. 6.10 Data--dependent allocation. 6.11 Trial designs other than two parallel groups. 6.12 Other aspects of drug development. 6.13 Further reading. 6.14 Key points. Exercises. 7. Observational Studies. 7.1 Introduction. 7.2 Alternative study designs. 7.3 Explicit modelling of biases. 7.4 Institutional comparisons. 7.5 Key points. Exercises. 8. Evidence Synthesis. 8.1 Introduction. 8.2 'Standard' meta--analysis. 8.2.1 A Bayesian perspective. 8.2.2 Some delicate issues in Bayesian meta--analysis. 8.2.3 The relationship between treatment effect and underlying risk. 8.3 Indirect comparison studies. 8.4 Generalised evidence synthesis. 8.5 Further reading. 8.6 Key points. Exercises. 9. Cost--effectiveness, Policy--Making and Regulation. 9.1 Introduction. 9.2 Contexts. 9.3 'Standard' cost--effectiveness analysis without uncertainty. 9.4 'Two--stage' and integrated approaches to uncertainty in cost--effectiveness modeling. 9.5 Probabilistic analysis of sensitivity to uncertainty about parameters: two--stage approach. 9.6 Cost--effectiveness analyses of a single study: integrated approach. 9.7 Levels of uncertainty in cost--effectiveness models. 9.8 Complex cost--effectiveness models. 9.8.1 Discrete--time, discrete--state Markov models. 9.8.2 Micro--simulation in cost--effectiveness models. 9.8.3 Micro--simulation and probabilistic sensitivity analysis. 9.8.4 Comprehensive decision modeling. 9.9 Simultaneous evidence synthesis and complex cost--effectiveness modeling. 9.9.1 Generalised meta--analysis of evidence. 9.9.2 Comparison of integrated Bayesian and two--stage approach. 9.10 Cost--effectiveness of carrying out research: payback models. 9.10.1 Research planning in the public sector. 9.10.2 Research planning in the pharmaceutical industry. 9.10.3 Value of information. 9.11 Decision theory in cost--effectiveness analysis, regulation and policy. 9.12 Regulation and health policy. 9.12.1 The regulatory context. 9.12.2 Regulation of pharmaceuticals. 9.12.3 Regulation of medical devices. 9.13 Conclusions. 9.14 Key points. Exercises. 10. Conclusions and Implications for Future Research. 10.1 Introduction. 10.2 General advantages and problems of a Bayesian approach. 10.3 Future research and development. Appendix: Websites and Software. A.1 The site for this book. A.2 Bayesian methods in health--care evaluation. A.3 Bayesian software. A.4 General Bayesian sites. References. Index.

Bayesian approaches to clinical trials and health-care evaluation

We examine the use of randomization-based inference for analyzing multiarmed randomized clinical trials, including the application of conditional randomization tests to multiple comparisons. The view is taken that the linkage of the statistical test to the experimental design (randomization procedure) should be recognized. A selected collection of randomization procedures generalized to multiarmed treatment allocation is summarized, and generalizations for two randomization procedures that heretofore were designed for only two treatments are developed. We explain the process of computing the randomization test and conditional randomization test via Monte Carlo simulation, developing an efficient algorithm that makes multiple comparisons possible that would not be possible using a standard algorithm, demonstrate the preservation of type I error rate, and explore the relationship of statistical power to the randomization procedure in the presence of a time trend and outliers. We distinguish between the interpretation of the p-value in the randomization test and in the population test and verify that the randomization test can be approximated by the population test on some occasions. Data from two multiarmed clinical trials from the literature are reanalyzed to illustrate the methodology.

Randomization tests for multiarmed randomized clinical trials.

A psychophysical method was used to estimate the timing of perception of kinesthetic stimuli with different velocities in normal volunteers. A 1 ms auditory click occurred randomly before or after an imposed flexion movement at either 20, 40 or 60 deg/s of the metacarpophalangeal joint. Subjects reported whether the click was perceived before or after the movement onset (experiment 1) or perception of movement velocity (experiment 2). The time at which there was a 50% chance that subjects reported movement or velocity perception after the click was taken as an estimate of the time subjects perceived the stimuli. The difference in time of perceived movement velocity discrimination and movement onset was only significant when the velocity was 20 deg/s (52 ms). This suggests that movement onset and identification of the velocity of the faster movements are perceived nearly simultaneously.

Perception of timing of kinesthetic stimuli.

In this article, we explore conditions under which the nonparametric randomization-based version of the logrank test is asymptotically normal for a wide class of randomization procedures.

On asymptotic normality of the randomization-based logrank test

Randomization and the Clinical Trial

A two-stage adaptive optimal design is an attractive option for increasing the efficiency of clinical trials. In these designs, based on interim data, the locally optimal dose is chosen for further exploration, which induces dependencies between data from the two stages. When the maximum likelihood estimator (MLE) is used under nonlinear regression models with independent normal errors in a pilot study where the first stage sample size is fixed, and the second stage sample size is large, the Fisher information fails to normalize the estimator adequately asymptotically, because of dependencies. In this situation, we present three alternative random information measures and show that they provide better normalization of the MLE asymptotically. The performance of random information measures is investigated in simulation studies, and the results suggest that the observed information performs best when the sample size is small.

William F. Rosenberger

Papers

Randomization tests for multiarmed randomized clinical trials.

Perception of timing of kinesthetic stimuli.

On asymptotic normality of the randomization-based logrank test

Randomization and the Clinical Trial

Random Norming Aids Analysis of Non-linear Regression Models with Sequential Informative Dose Selection.