Showing papers on "Sample size determination published in 1992"

A power primer.

Abstract: One possible reason for the continued neglect of statistical power analysis in research in the behavioral sciences is the inaccessibility of or difficulty with the standard material. A convenient, although not comprehensive, presentation of required sample sizes is provided here. Effect-size indexes and conventional values for these are given for operationally defined small, medium, and large effects. The sample sizes necessary for .80 power to detect effects at these levels are tabled for eight standard statistical tests: (a) the difference between independent means, (b) the significance of a product-moment correlation, (c) the difference between independent rs, (d) the sign test, (e) the difference between independent proportions, (f) chi-square tests for goodness of fit and contingency tables, (g) one-way analysis of variance, and (h) the significance of a multiple or multiple partial correlation.

Designing and Conducting Survey Research: A Comprehensive Guide

Abstract: Figures, Tables, Exhibits, and Worksheets. Preface. The Authors. PART ONE: DEVELOPING AND ADMINISTERING QUESTIONNAIRES. 1. An Overview of the Sample Survey Process. 2. Designing Effective Questionnaires: Basic Guidelines. 3. Developing Survey Questions. 4. Utilizing Focus Groups in the Survey Research Process. PART TWO: ENSURING SCIENTIFIC ACCURACY. 5. Descriptive Statistics: Measures of Central Tendency and Dispersion. 6. The Theoretical Basis of Sampling. 7. Confidence Intervals and Basic Hypothesis Testing. 8. Determining the Sample Size. 9. Selecting a Representative Sample. PART THREE: PRESENTING AND ANALYZING SURVEY RESULTS. 10. Analyzing Cross-Tabulated Data. 11. Testing the Difference Between Means. 12. Regression and Correlation. 13. Preparing an Effective Final Report. Resource A: Table of Areas of a Standard Normal Distribution. Resource B: Glossary. Resource C: Answers to Selected Exercises. Bibliography. Index.

Can test statistics in covariance structure analysis be trusted

Abstract: Covariance structure analysis uses chi 2 goodness-of-fit test statistics whose adequacy is not known. Scientific conclusions based on models may be distorted when researchers violate sample size, variate independence, and distributional assumptions. The behavior of 6 test statistics is evaluated with a Monte Carlo confirmatory factor analysis study. The tests performed dramatically differently under 7 distributional conditions at 6 sample sizes. Two normal-theory tests worked well under some conditions but completely broke down under other conditions. A test that permits homogeneous nonzero kurtoses performed variably. A test that permits heterogeneous marginal kurtoses performed better. A distribution-free test performed spectacularly badly in all conditions at all but the largest sample sizes. The Satorra-Bentler scaled test statistic performed best overall.

QUANTITATIVE METHODS IN PSYCHOLOGY A Power Primer

Abstract: One possible reason for the continued neglect of statistical power analysis in research in the behavioral sciences is the inaccessibility of or difficulty with the standard material. A convenient, although not comprehensive, presentation of required sample sizes is provided here. Effect-size indexes and conventional values for these are given for operationally defined small, medium, and large effects. The sample sizes necessary for .80 power to detect effects at these levels are tabled for eight standard statistical tests: (a) the difference between independent means, (b) the significance of a product-moment correlation, (c) the difference between independent rs, (d) the sign test, (e) the difference between independent proportions, (f) chi-square tests for goodness of fit and contingency tables, (g) one-way analysis of variance, and (h) the significance o f a multiple or multiple partial correlation. The preface to the first edition of my power handbook (Cohen, 1969) begins: During my first dozen years o f teaching and consulting o n applied statistics with behavioral scientists, 1 became increasingly impressed with the importance of statistical power analysis, an importance which was increased an order of magnitude by its neglect in our textbooks and curricula. The case for its importance is easily made: What behavioral scientist would view with equanimity the question of the probability that his investigation would lead to statistically significant results, i.e., its power? (p. vii) This neglect was obvious through casual observation and had been confirmed by a power review of the 1960 volume of the Journal of Abnormal and Social Psychology, which found the mean power to detect medium effect sizes to be .48 (Cohen, 1962). Thus, the chance of obtaining a significant result was about that of tossing a head with a fair coin. I attributed this disregard of power to the inaccessibility of a meager and mathematically difficult literature, beginning with its origin in the work of Neyman and Pearson (1928,1933). The power handbook was supposed to solve the problem. It required no more background than an introductory psychological statistics course that included significance testing. The exposition was verbal-intuitive and carried largely by many worked examples drawn from across the spectrum of behavioral science. In the ensuing two decades, the book has been through revised (1977) and second (1988) editions and has inspired dozens of power and effect-size surveys i n many areas of the social and life sciences (Cohen, 1988, pp. xi-xii). During this period, there has been a spate of articles on power analysis in the social science literature, a baker's dozen of computer programs (re

Measurement of motor recovery after stroke. Outcome assessment and sample size requirements.

Abstract: The purpose of this study was to analyze recovery of motor function in a cohort of patients presenting with an acute occlusion in the carotid distribution. Analysis of recovery patterns is important for estimating patient care needs, establishing therapeutic plans, and estimating sample sizes for clinical intervention trials.We prospectively measured the motor deficits of 104 stroke patients over a 6-month period to identify earliest measures that would predict subsequent motor recovery. Motor function was measured with the Fugl-Meyer Assessment. Fifty-four patients were randomly assigned to a training set for model development; 50 patients were assigned to a test set for model validation. In a second analysis, patients were stratified on basis of time and stroke severity. The sample size required to detect a 50% improvement in residual motor function was calculated for each level of impairment and at three points in time.At baseline the initial Fugl-Meyer motor scores accounted for only half the variance...

Repeated measures in clinical trials: Analysis using mean summary statistics and its implications for design

Abstract: This paper explores the use of simple summary statistics for analysing repeated measurements in randomized clinical trials with two treatments. Quite often the data for each patient may be effectively summarized by a pre-treatment mean and a post-treatment mean. Analysis of covariance is the method of choice and its superiority over analysis of post-treatment means or analysis of mean changes is quantified, as regards both reduced variance and avoidance of bias, using a simple model for the covariance structure between time points. Quantitative consideration is also given to practical issues in the design of repeated measures studies: the merits of having more than one pre-treatment measurement are demonstrated, and methods for determining sample sizes in repeated measures designs are provided. Several examples from clinical trials are presented, and broad practical recommendations are made. The examples support the value of the compound symmetry assumption as a realistic simplification in quantitative planning of repeated measures trials. The analysis using summary statistics makes no such assumption. However, allowance in design for alternative non-equal correlation structures can and should be made when necessary.

A comparison of some methodologies for the factor analysis of non‐normal Likert variables: A note on the size of the model

Abstract: This paper expands on a recent study by Muthen & Kaplan (1985) by examining the impact of non-normal Likert variables on testing and estimation in factor analysis for models of various size. Normal theory GLS and the recently developed ADF estimator are compared for six cases of non-normality, two sample sizes, and four models of increasing size in a Monte Carlo framework with a large number of replications. Results show that GLS and ADF chi-square tests are increasingly sensitive to non-normality when the size of the model increases. No parameter estimate bias was observed for GLS and only slight parameter bias was found for ADF. A downward bias in estimated standard errors was found for GLS which remains constant across model size. For ADF, a downward bias in estimated standard errors was also found which became increasingly worse with the size of the model.

Sample adequately to estimate variograms of soil properties

Abstract: SUMMARY The variogram is central in the spatial analysis of soil, yet it is often estimated from few data, and its precision is unknown because confidence limits cannot be determined analytically from a single set of data. Approximate confidence intervals for the variogram of a soil property can be found numerically by simulating a large field of values using a plausible model and then taking many samples from it and computing the observed variogram of each sample. A sampling distribution of the variogram and its percentiles can then be obtained. When this is done for situations typical in soil and environmental surveys it seems that variograms computed on fewer than 50 data are of little value and that at least 100 data are needed. Our experiments suggest that for a normally distributed isotropic variable a variogram computed from a sample of 150 data might often be satisfactory, while one derived from 225 data will usually be reliable.

Summarizing Monte Carlo results in methodological research: The one- and two-factor fixed effects ANOVA cases.

Abstract: Meta-analytic methods were used to integrate the findings of a sample of Monte Carlo studies of the robustness of the F test in the one- and two-factor fixed effects ANOVA models. Monte Carlo results for theWelch (1947) and Kruskal-Wallis (Kruskal & Wallis, 1952) tests were also analyzed. The meta-analytic results provided strong support for the robustness of the Type I error rate of the F test when certain assumptions were violated. The F test also showed excellent power properties. However, the Type I error rate of the F test was sensitive to unequal variances, even when sample sizes were equal. The error rate of the Welch test was insensitive to unequal variances when the population distribution was normal, but nonnormal distributions tended to inflate its error rate and to depress its power. Meta-analytic and exact statistical theory results were used to summarize the effects of assumption violations for the tests.

Estimating population size for capture-recapture data when capture probabilities vary by time and individual animal.

Abstract: There have been no estimators of population size associated with the capture-recapture model when the capture probabilities vary by time and individual animal. This work proposes a nonparametric estimation technique that is appropriate for such a model using the idea of sample coverage, which is defined as the proportion of the total individual capture probabilities of the captured animals. A simulation study was carried out to show the performance of the proposed estimation procedure. Numerical results indicate that it generally works satisfactorily when the coefficient of variation of the individual capture probabilities is relatively large. An example is also given for illustration.

Jackknife variance estimation with survey data under hot deck imputation

Abstract: SUMMARY Hot deck imputation is commonly employed for item nonresponse in sample surveys. It is also a common practice to treat the imputed values as if they are true values, and then compute the variance estimates using standard formulae. This procedure, however, could lead to serious underestimation of the true variance, when the proportion of missing values for an item is appreciable. We propose a jackknife variance estimator for stratified multistage surveys which is obtained by first adjusting the imputed values for each pseudo-replicate and then applying the standard jackknife formula. The proposed jackknife variance estimator is shown to be consistent as the sample size increases, assuming equal response probabilities within imputation classes and using a particular hot deck imputation.

Measurement Error and Morphometric Studies: Statistical Power and Observer Experience

Abstract: We estimated measurement error (ME) associated with 15 skeletal characters in seven species of passerine birds. The total sample variance of the characters, from three mea? surements of each character on each individual, was partitioned into among- and within-indi? vidual components of variation. Measurement error, defined as that portion of the total sample variance made up by within-individual variation, ranged from 80%. Percent ME was inversely related to the size of a character, among-individual variation, and experience of the measurer. Other sources of ME included the character's flexibility and the definition of character landmarks. Although observer experience reduced ME, the effect was only substantial after more specimens were measured than are often used in morphometric studies. The magnitude of statistically detectable differences between groups was examined in relation to the level of ME and the sample sizes of groups. Measurement error increased the chances of finding no group difference, i.e., making type II errors. Type II errors can be reduced either by averaging two or more repeated measurements of the same individual, thereby reducing ME, or by increasing the sample size. We show how averaging repeated measurements changes the minimum detectable effect size between groups when both ME and sample size vary. The pattern of character- specific measurement error is consistent across passerine species; both reliable and unreliable characters for morphometric studies are identified. (Measurement error; morphometrics; type II error; passerines; model II ANO VA; statistical power.)

Assessing Effects of Unreplicated Perturbations: No Simple Solutions

Abstract: We address the task of determining the effects, on mean population density or other parameters, of an unreplicated perturbation, such as arises in environmental assessments and some ecosystem-level experiments. Our context is the Before-After-Con- trol-Impact-Pairs design (BACIP): on several dates Before and After the perturbation, samples are collected simultaneously at both the Impact site and a nearby "Control." One approach is to test whether the mean of the Impact-Control difference has changed from Before to After the perturbation. If a conventional test is used, checks of its as- sumptions are an important and messy part of the analysis, since BACIP data do not necessarily satisfy them. It has been suggested that these checks are not needed for ran- domization tests, because they are insensitive to some of these assumptions and can be adjusted to allow for others. A major aim of this paper is to refute this suggestion: there is no panacea for the difficult and messy technical problems in the analysis of data from assessments or unreplicated experiments. We compare the randomization t test with the standard t test and the modified (Welch- Satterthwaite-Aspin) t test, which allows for unequal variances. We conclude that the randomization t test is less likely to yield valid inferences than is the Welch t test, because it requires identical distributions for small sample sizes and either equal variances or equal sample sizes for larger ones. The formal requirement of Normality is not crucial to the Welch t test. Both parametric and randomization tests require that time and location effects be additive and that Impact-Control differences on different dates be independent. These assumptions should be tested; if they are seriously wrong, alternative analyses are needed. This will often require a long time series of data. Finally, for assessing the importance of a perturbation, the P value of a hypothesis test is rarely as useful as an estimate of the size of the effect. Especially if effect size varies with time and conditions, flexible estimation methods with approximate answers are preferable to formally exact P values.

On One-Step GM Estimates and Stability of Inferences in Linear Regression

Abstract: The folklore on one-step estimation is that it inherits the breakdown point of the preliminary estimator and yet has the same large sample distribution as the fully iterated version as long as the preliminary estimate converges faster than n –1/4, where n is the sample size. We investigate the extent to which this folklore is valid for one-step GM estimators and their associated standard errors in linear regression. We find that one-step GM estimates based on Newton-Raphson or Scoring inherit the breakdown point of high breakdown point initial estimates such as least median of squares provided the usual weights that limit the influence of extreme points in the design space are based on location and scatter estimates with high breakdown points. Moreover, these estimators have bounded influence functions, and their standard errors can have high breakdown points. The folklore concerning the large sample theory is correct assuming the regression errors are symmetrically distributed and homoscedastic....

On the Removal of Skewness by Transformation

Abstract: SUMMARY We suggest empirical transformations for removing most of the skewness of an asymmetric statistic. These transformations may be used as the basis for accurate confidence procedures or hypothesis tests and can be employed in conjunction with either a normal approximation or the bootstrap. Our approach differs from those of other researchers in that the transformation is monotone and invertible. In this paper we suggest elementary but general transformations for eliminating skewness from the distribution of a Studentized statistic. The transformations are monotone and have simple, explicit inversion formulae, and so are readily applied to confidence interval problems. When the transformations are used in conjunction with the bootstrap, they can produce confidence intervals with particularly low levels of coverage error. That confidence intervals for symmetric statistics have lower coverage error than their counterparts in the case of asymmetry may be deduced from Edgeworth expansions. The first term in an expansion, of size n - 1/2 where n is the sample size, describes the error in the usual normal approximation and is due entirely to skewness. If it can be eliminated, for example by transformation, then the normal approxima- tion will be in error by only O(n- 1). Thus, our method involves transforming one statistic into another, where the distribution is virtually symmetric, applying the normal approximation (or the bootstrap) to the new statistic and then regaining the asymmetry of the original problem by inverse transformation, at the same time retain- ing the high coverage accuracy conferred by applying confidence interval procedures to a symmetric statistic.

Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance

Abstract: Monitoring clinical trials in nonfatal diseases where ethical considerations do not dictate early termination upon demonstration of efficacy often requires examining the interim findings to assure that the protocol-specified sample size will provide sufficient power against the null hypothesis when the alternative hypothesis is true. The sample size may be increased, if necessary to assure adequate power. This paper presents a new method for carrying out such interim power evaluations for observations from normal distributions without unblinding the treatment assignments or discernably affecting the Type 1 error rate. Simulation studies confirm the expected performance of the method.

Testing Goodness-of-Fit in Regression Via Order Selection Criteria

Abstract: A new test is derived for the hypothesis that a regression function has a prescribed parametric form. Unlike many recent proposals, this test does not depend on arbitrarily chosen smoothing parameters. In fact, the test statistic is itself a smoothing parameter which is selected to minimize an estimated risk function. The exact distribution of the test statistic is obtained when the error terms in the regression model are Gaussian, while the large sample distribution is derived for more general settings. It is shown that the proposed test is consistent against fixed alternatives and can detect local alternatives that converge to the null hypothesis at the rate $1/\sqrt n$, where $n$ is the sample size. More importantly, the test is shown by example to have an ability to adapt to the alternative at hand.

A goodness‐of‐fit approach to inference procedures for the kappa statistic: Confidence interval construction, significance‐testing and sample size estimation

Abstract: We propose a new procedure for constructing a confidence interval about the kappa statistic in the case of two raters and a dichotomous outcome. The procedure is based on a chi-square goodness-of-fit test as applied to a model frequently used for clustered binary data. The procedure provides coverage levels that are accurate in samples of smaller size than those required for other procedures. The procedure also has use for significance-testing and the planning of corresponding sample size requirements.

Testing the autocorrelation structure of disturbances in ordinary least squares and instrumental variables regressions

Abstract: This paper derives the asymptotic distribution for a vector of sample autocorrelations of regression residuals from a quite general linear model. The asymptotic distribution forms the basis for a test of the null hypothesis that the regression error follows a moving average of order q [greaterthan or equal] 0 against the general alternative that autocorrelations of the regression error are non-zero at lags greater than q. By allowing for endogenous, predetermined and/or exogenous regressors, for estimation by either ordinary least squares or a number of instrumental variables techniques, for the case q>0, and for a conditionally heteroscedastic error term, the test described here is applicable in a variety of situations where such popular tests as the Box-Pierce (1970) test, Durbin's (1970) h test, and Godfrey's (1978b) Lagrange multiplier test are net applicable. The finite sample properties of the test are examined in Monte Carlo simulations where, with a sample sizes of 50 and 100 observations, the test appears to be quite reliable.

Sixteen S-squared over D-squared: A relation for crude sample size estimates

Abstract: I suggest for memorization an equation for calculating approximate sample size requirements intended only for a specific set of values (80 per cent power for a two-tailed alpha = 0.05 test) which seems to occur often in biopharmaceutical research. After presenting the formula in terms of variance estimate s2 and effect size d, I derive a few alternative forms and then discuss the accuracy of the approximation and other properties as well as examples of its use.

Sixteen S squared over D squared: A relation for crude sample size estimates

Abstract: I suggest for memorization an equation for calculating approximate sample size requirements intended only for a specific set of values (80 per cent power for a two-tailed alpha = 0.05 test) which seems to occur often in biopharmaceutical research. After presenting the formula in terms of variance estimate s2 and effect size d, I derive a few alternative forms and then discuss the accuracy of the approximation and other properties as well as examples of its use.

The effect of trial size on statistical power.

Abstract: Many research studies produce results that falsely support a null hypothesis due to a lack of statistical power. The purpose of this research was to demonstrate selected relationships between single subject (SS) and group analyses and the importance of data reliability (trial size) on results. A computer model was developed and used in conjunction with Monte Carlo procedures to study the effects of sample size (subjects and trials), within- and between-subject variability, and subject performance strategies on selected statistical evaluation procedures. The inherent advantages of the approach are control and replication. Selected results are presented in this paper. Group analyses on subjects using similar performance strategies identified 10, 5, and 3 trials for sample sizes of 5, 10, and 20, respectively, as necessary to achieve statistical power values greater than 90% for effect sizes equal to one standard deviation of the condition distribution. SS analyses produced results exhibiting considerably less power than the group results for corresponding trial sizes, indicating how much more difficult it is to detect significant differences using a SS design. These results should be of concern to all investigators especially when interpreting nonsignificant findings.

Interim analyses for monitoring clinical trials that do not materially affect the type I error rate.

Abstract: Monitoring clinical trials often requires examining the interim findings to see if the sample size originally specified in the protocol will provide the required power against the null hypothesis when the alternative hypothesis is true, and to increase the sample size if necessary. This paper presents a new method, based on the overall response rate, for carrying out interim power evaluations when the observations have binomial distributions, without unblinding the treatment assignments or materially affecting the type I error rate. Simulation study results confirm the performance of the method.

“Equivalent Sample Size” and “Equivalent Degrees of Freedom” Refinements for Inference Using Survey Weights under Superpopulation Models

Abstract: A number of procedures have been proposed to attack different inference problems for data drawn from a survey with a complex sample design (i.e., a design that entails unequal weighting). Most procedures either are based on finite-population assumptions or require the specification of an explicit model using a superpopulation rationale. Herein we propose some relatively simple approximate procedures that are based on a superpopulation model. They provide valid variance estimators, test statistics, and confidence intervals that allow for sample design effects as expressed by design weights and other weights. The procedures do not rely on conditioning on model elements such as covariates to adjust for design effects. Instead, we obtain estimators by rescaling sample weights to sum to the equivalent sample size (equal to sample size divided by design effect). Using weighted estimators for superpopulation models, we obtain approximations to confidence bounds on the mean for simple sampling situations...

Sample size requirements for addressing the population genetic issues of forensic use of DNA typing.

Abstract: DNA typing offers a unique opportunity to identify individuals for medical and forensic purposes. Probabilistic inference regarding the chance occurrence of a match between the DNA type of an evidentiary sample and that of an accused suspect, however, requires reliable estimation of genotype and allele frequencies in the population. Although population-based data on DNA typing at several hypervariable loci are being accumulated at various laboratories, a rigorous treatment of the sample size needed for such purposes has not been made from population genetic considerations. It is shown here that the loci that are potentially most useful for forensic identification of individuals have the intrinsic property that they involve a large number of segregating alleles, and a great majority of these alleles are rare. As a consequence, because of the large number of possible genotypes at the hypervariable loci that offer the maximum potential for individualization, the sample size needed to observe all possible genotypes in a sample is large. In fact, the size is so large that even if such a huge number of individuals could be sampled, it could not be guaranteed that such a sample was drawn from a single homogeneous population. Therefore adequate estimation of genotypic probabilities must be based on allele frequencies, and the sample size needed to represent all possible alleles is far more reasonable. Further economization of sample size is possible if one wants to have representation of only the frequent alleles in the sample, so that the rare allele frequencies can be approximated by an upper bound for forensic applications.

Normal Goodness-of-Fit Tests for Multinomial Models with Large Degrees of Freedom

Abstract: Goodness-of-fit tests for independent multinomials with parameters to be estimated are usually based on Pearson's X 2 or the likelihood ratio G 2. Both are included in the family of power-divergence statistics SD λ. For increasing sample sizes each SD λ has an asymptotic X 2 distribution, provided that the number of cells remains fixed. We are dealing with an increasing-cells approach, where the number J of independent multinomials increases while the number of classes for each multinomial and the number of parameters remain fixed. Extending results on X 2 and G 2, the asymptotic normality of any SD λ is obtained for increasing cells. The corresponding normal goodness-of-fit tests discussed here apply for models with large degrees of freedom with no restrictions imposed on the sizes N j of each multinomial, allowing large as well as small expectations (sparse data) within each cell. The asymptotic expectation and variance of SD λ are easy to compute for Pearson's X 2 and simplify considerably for...

Sample size requirements for stratified cluster randomization designs

Abstract: Sample size requirements are provided for designs of studies in which clusters are randomized within each of several strata, where cluster size itself may be a stratifying factor. The approach generalizes a formula derived by Woolson et al., which provides sample size requirements for the Cochran-Mantel-Haenszel statistic. Issues of data analysis are also discussed.