Showing papers on "Statistical hypothesis testing published in 1995"

Bayesian Model Selection in Social Research

Abstract: It is argued that P-values and the tests based upon them give unsatisfactory results, especially in large samples. It is shown that, in regression, when there are many candidate independent variables, standard variable selection procedures can give very misleading results. Also, by selecting a single model, they ignore model uncertainty and so underestimate the uncertainty about quantities of interest. The Bayesian approach to hypothesis testing, model selection, and accounting for model uncertainty is presented. Implementing this is straightforward through the use of the simple and accurate BIC approximation, and it can be done using the output from standard software. Specific results are presented for most of the types of model commonly used in sociology. It is shown that this approach overcomes the difficulties with P-values and standard model selection procedures based on them. It also allows easy comparison of nonnested models, and permits the quantification of the evidence for a null hypothesis of interest, such as a convergence theory or a hypothesis about societal norms.

Detecting earnings management

Abstract: This paper evaluates alternative accrual-based models for detecting earnings management. The evaluation compares the specification and power of commonly used test statistics across the measures of discretionary accruals generated by the models and provides the following major insights. First, all of the models appear well specified when applied to a random sample of firm-years. Second, the models all generate tests of low power for earnings management of economically plausible magnitudes (e.g., one to five percent of total assets). Third, all models reject the null hypothesis of no earnings management at rates exceeding the specified test-levels when applied to samples of firms with extreme financial performance. This result highlights the importance of controlling for financial performance when investigating earnings management stimuli that are correlated with financial performance. Finally, a modified version of the model developed by Jones (1991) exhibits the most power in detecting earnings management.

Likelihood-Based Inference in Cointegrated Vector Autoregressive Models

Abstract: This book gives a detailed mathematical and statistical analysis of the cointegrated vector autoregresive model This model had gained popularity because it can at the same time capture the short-run dynamic properties as well as the long-run equilibrium behaviour of many non-stationary time series It also allows relevant economic questions to be formulated in a consistent statistical framework Part I of the book is planned so that it can be used by those who want to apply the methods without going into too much detail about the probability theory The main emphasis is on the derivation of estimators and test statistics through a consistent use of the Guassian likelihood function It is shown that many different models can be formulated within the framework of the autoregressive model and the interpretation of these models is discussed in detail In particular, models involving restrictions on the cointegration vectors and the adjustment coefficients are discussed, as well as the role of the constant and linear drift In Part II, the asymptotic theory is given the slightly more general framework of stationary linear processes with iid innovations Some useful mathematical tools are collected in Appendix A, and a brief summary of weak convergence in given in Appendix B The book is intended to give a relatively self-contained presentation for graduate students and researchers with a good knowledge of multivariate regression analysis and likelihood methods The asymptotic theory requires some familiarity with the theory of weak convergence of stochastic processes The theory is treated in detail with the purpose of giving the reader a working knowledge of the techniques involved Many exercises are provided The theoretical analysis is illustrated with the empirical analysis of two sets of economic data The theory has been developed in close contract with the application and the methods have been implemented in the computer package CATS in RATS as a result of a rcollaboation with Katarina Juselius and Henrik Hansen

Multiple Hypothesis Testing

Abstract: INTRODUCTION ..................................................................................................................... 561 ORGANIZING CO CEPTS ..................................................................................................... 564 Primary Hypotheses, Closure, Hierarchical Sets, and Minimal Hypotheses ...................... 564 Families ................................................................................................................................ 565 Type 1 Error Control ............................................................................................................ 566 Power ................................................................................................................................... 567 P-Values and Adjusted P-Values ......................................................................................... 568 Closed Test Procedures ....................................................................................................... 569 METHODS BA ED ON ORDERED P-VALUES ................................................................... 569 Methods Based on the First-Order Bonferroni Inequality .................................................. 569 Methods Based on the Simes Equality ................................................................................. 570 Modifications for Logically Related Hypotheses ................................................................. 571 Methods Controlling the False Discovery Rate ................................................................... 572 COMPARING NORMALLY DISTRIBUTED M ANS ......................................................... 573 OTHER ISSUES ........................................................................................................................ 575 Tests vs Confidence I tervals ............................................................................................... 575 Directional vs Nondirectional Inference ............................................................................. 576 Robustness ............................................................................................................................ 577 Others ........................................................................ ....................................................... 578 CONCLUSION .......................................................................................................................... 580

Measurement Error in Nonlinear Models

Abstract: Preface Guide to Notation 1. Introduction 2. Regression and Attenuation 3. Regression Calibration 4. Simulation Extrapolation 5. Instrumental Variables 6. Functional Methods 7. Likelihood and Quasilikelihood 8. Bayesian Methods 9. Semiparametric Methods 10. Unknown Link Functions 11. Hypothesis Testing 12. Density Estimation and Nonparametric Regression 13. Response Variable Error 14. Other Topics Appendix: Fitting Methods and Models References Author Index Subject Index

Statistical Methods in the Atmospheric Sciences: An Introduction

Abstract: Introduction Review of Probability Empirical Distributions and Exploratory Data Analysis Theoretical Probability Distributions Hypothesis Testing Statistical Weather Forecasting Forecast Verification Time Series Methodsfor Multivariate Data Chapter Exercises Appendices: Example Data Sets Selected Probability Tables Answers to Exercises References Subject Index

Comparative analysis by independent contrasts (CAIC): an Apple Macintosh application for analysing comparative data.

Abstract: CAIC is an application for the Apple Macintosh which allows the valid analysis of comparative (multi-species) data sets that include continuous variables. Comparison among species is the most common technique for testing hypotheses of how organisms are adapted to their environments, but standard statistical tests like regression should not be used with species data. Such tests assume independence of data points, but related species often share traits by common descent rather than through independent adaptation. CAIC uses a phylogeny of the species in the data set to partition the variance among species into independent comparisons (technically, linear contrasts), each comparison being made at a different node in the phylogeny. There are two partitioning procedures--one used when all variables are continuous, the other when one variable is discrete. The resulting comparisons can be analysed validly in standard statistical packages to test hypotheses about correlated evolution among traits, to estimate parameters such as allometric exponents, and to compare rates of evolution. Previous versions of the package have already been used widely; this version is simpler to use and works on a wider range of machines. The package and manual are freely available by anonymous ftp or from the authors.

Empirical Methods for Artificial Intelligence

Abstract: Empirical research exploratory data analysis basic issues in experiment design hypothesis testing and estimation computer-intensive statistical methods performance assessment explaining performance - interactions and dependencies modelling tactics for generalization.

An E-M algorithm and testing strategy for multiple-locus haplotypes.

Abstract: This paper gives an expectation maximization (EM) algorithm to obtain allele frequencies, haplotype frequencies, and gametic disequilibrium coefficients for multiple-locus systems. It permits high polymorphism and null alleles at all loci. This approach effectively deals with the primary estimation problems associated with such systems; that is, there is not a one-to-one correspondence between phenotypic and genotypic categories, and sample sizes tend to be much smaller than the number of phenotypic categories. The EM method provides maximum-likelihood estimates and therefore allows hypothesis tests using likelihood ratio statistics that have chi 2 distributions with large sample sizes. We also suggest a data resampling approach to estimate test statistic sampling distributions. The resampling approach is more computer intensive, but it is applicable to all sample sizes. A strategy to test hypotheses about aggregate groups of gametic disequilibrium coefficients is recommended. This strategy minimizes the number of necessary hypothesis tests while at the same time describing the structure of disequilibrium. These methods are applied to three unlinked dinucleotide repeat loci in Navajo Indians and to three linked HLA loci in Gila River (Pima) Indians. The likelihood functions of both data sets are shown to be maximized by the EM estimates, and the testing strategy provides a useful description of the structure of gametic disequilibrium. Following these applications, a number of simulation experiments are performed to test how well the likelihood-ratio statistic distributions are approximated by chi 2 distributions. In most circumstances the chi 2 grossly underestimated the probability of type I errors. However, at times they also overestimated the type 1 error probability. Accordingly, we recommended hypothesis tests that use the resampling method.

Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio

Abstract: We present a simple method for computing Bayes factors. The method derives from observing that in general, a Bayes factor can be written as the product of a quantity called the Savage-Dickey density ratio and a correction factor; both terms are easily estimated from posterior simulation. In some cases it is possible to do these computations without ever evaluating the likelihood.

Publication decisions revisited: the effect of the outcome of statistical tests on the decision to publish and vice versa

Abstract: This article presents evidence that published results of scientific investigations are not a representative sample of results of all scientific studies. Research studies from 11 major journals demonstrate the existence of biases that favor studies that observe effects that, on statistical evaluation, have a low probability of erroneously rejecting the so-called null hypothesis (H 0). This practice makes the probability of erroneously rejecting H 0 different for the reader than for the investigator. It introduces two biases in the interpretation of the scientific literature: one due to multiple repetition of studies with false hypothesis, and one due to failure to publish smaller and less significant outcomes of tests of a true hypotheses. These practices distort the results of literature surveys and of meta-analyses. These results also indicate that practice leading to publication bias have not changed over a period of 30 years.

A General Purpose Approximate Goodness-of-Fit Test

Abstract: Skewed distributions play an important role in the analysis of data from quality and reliability experiments Very often unknown parameters must be estimated from the sample data in order to test whether the data has come from a certain family of distri

A Stastistical Analysis of Cointegration for I(2) Variables

Abstract: This paper discusses inference for I(2) variables in a VAR model. The estimation procedure suggested consists of two reduced rank regressions. The asymptotic distribution of the proposed estimators of the cointegrating coefficients is mixed Gaussian, which implies that asymptotic inference can be conducted using the χ2 distribution. It is shown to what extent inference on the cointegration ranks can be conducted using the tables already prepared for the analysis of cointegration of I(1) variables. New tables are needed for the test statistics to control the size of the tests. This paper contains a multivariate test for the existence of I(2) variables. This test is illustrated using a data set consisting of U.K. and foreign prices and interest rates as well as the exchange rate.

Maximum likelihood trees from dna sequences: a peculiar statistical estimation problem

Abstract: The parameter space of the phylogenetic tree estimation problem consists of three com? ponents, T, t, and 8. The tree topology T is a discrete entity that is not a proper statistical parameter but that can nevertheless be estimated using the maximum likelihood criterion. Its role is to specify the branch length parameters and the form of the likelihood function(s). Branch lengths t are conditional on T and are meaningful only for specific values of T. Parameters 8 in the model of nucleotide substitution are common to all the tree topologies and represent such values as the transition/trans version rate ratio. T and t thus represent the tree, and 8 represents the model. With typical DNA sequence data, differences in T have only a small effect on the likelihood, but changing 8 will influence the likelihood greatly. Estimates of 8 are also found to be insensitive to T, making it possible to obtain reliable estimates of 8 and to perform tests concerning the model (8) even if knowledge of the evolutionary relationship (T) is not available. In contrast, tests con? cerning t, such as testing the existence of a molecular clock, appear to be more difficult to perform when the true topology is unknown. In this paper, we explore the peculiarity of the parameter space of the tree estimation problem and suggest methods for overcoming some difficulties in? volved with tests concerning the model. We also address difficulties concerning hypothesis testing on T, i.e., evaluation of the reliability of the estimated tree topology. We note that estimation of and particularly tests concerning T depend critically on the assumed model. (Maximum likeli? hood; models; parameter space; consistency; sampling errors; hypothesis testing; nucleotide sub? stitution; phylogeny estimation; molecular systematics; molecular clock.)

Intermediate Statistics and Econometrics: A Comparative Approach

Abstract: Basic concepts special distributions distributions of functions of random variables sampling theory estimation hypothesis testing prediction the linear regression model other windows on the world. Appendices: matrix algebra review I matrix algebra review II computation statistical tables.

Asymptotic Efficiency of Nonparametric Tests

Abstract: Making a substantiated choice of the most efficient statistical test is one of the basic problems of statistics. Asymptotic efficiency is an indispensable technique for comparing and ordering statistical tests in large samples. It is especially useful in nonparametric statistics where it is usually necessary to rely on heuristic tests. This monograph presents a unified treatment of the analysis and calculation of the asymptotic efficiencies of nonparametric tests. Powerful new methods are developed to evaluate explicitly different kinds of efficiencies. Of particular interest is the description of domains of the Bahadur local optimality and related characterisation problems based on recent research by the author. Other Russian results are also published here for the first time in English. Researchers, professionals and students in statistics will find this book invaluable.

Statistical Methods for Engineers and Scientists

Abstract: Probability distributions descriptive statistics expected values and mooments statistical inference - estimation, hypothesis testing analysis of variance regression analysis statistical progress control and reliability experimental design introduction to SAS tables of statistical functions answers to selected problems.

Nonlinear demographic dynamics: mathematical models, statistical methods, and biological experiments'

Abstract: Our approach to testing nonlinear population theory is to connect rigorously mathematical models with data by means of statistical methods for nonlinear time series. We begin by deriving a biologically based demographic model. The mathematical analysis identifies boundaries in parameter space where stable equilibria bifurcate to periodic 2-cy- cles and aperiodic motion on invariant loops. The statistical analysis, based on a stochastic version of the demographic model, provides procedures for parameter estimation, hypothesis testing, and model evaluation. Experiments using the flour beetle Tribolium yield the time series data. A three-dimensional map of larval, pupal, and adult numbers forecasts four possible population behaviors: extinction, equilibria, periodicities, and aperiodic motion including chaos. This study documents the nonlinear prediction of periodic 2-cycles in laboratory cultures of Tribolium and represents a new interdisciplinary approach to un- derstanding nonlinear ecological dynamics.

Objections to Bootstrapping Phylogenies: A Critique

Abstract: Despite widespread use, the bootstrap remains a controversial method for assessing confidence limits in phylogenies. Opposition to its use has centered on a small set of basic philo? sophical and statistical objections that have largely gone unanswered by advocates of statistical approaches to phylogeny reconstruction. The level of generality of these objections varies greatly, however. Some of the objections are merely technical, involving problems that are found in almost all statistical tests, such as bias in small data sets. Other objections are really associated not so much with a rejection of the bootstrap but with the rejection of statistical methods in phylogeny reconstruction, which resurrects an old debate. The most relevant aspects of this debate revolve around the issue of whether or not an unknown parameter, such as a tree, can have probabilities (confidence limits) associated with it. The relevant statistical aspects are reviewed, but because this issue remains controversial within statistical theory, it is unreasonable to expect it to be anything else in phylogenetic systematics. An area of common ground between statistical and nonstatistical approaches emerges in the use of statistical likelihood as a measure of support for phylogenetic hypotheses. This common ground requires the abandonment of classical notions of confidence limits by statistically oriented systematists and the acceptance of probabilistic models and likelihood by opponents of statistical methods. There remains a small set of objections directly germane to bootstrapping phylogenies per se. These objections involve issues of random sampling and whether or not character data are independent and identically distributed (HD). Nonrandom- sample bootstrapping is discussed, as are sample designs that impose the HD assumption on characters regardless of evolutionary nonindependence and nonidentical distribution of those data. Systematists wishing to use the bootstrap have an alternative to making explicit and rather strong evolutionary assumptions; they can consider the issue of character sampling designs much more carefully. (Phylogeny; bootstrap; statistical inference; confidence; cladistics.)

Statistical Inference via Bootstrapping for Measures of Inequality

Abstract: In this paper we consider the use of bootstrap methods to compute interval estimates and perform hypothesis tests for decomposable measures of economic inequality. The bootstrap potentially represents a significant gain over available asymptotic intervals because it provides an easily implemented solution to the Behrens-Fisher problem. Two applications of this approach, using the PSID (for the study of taxation) and the NLSY (for the study of youth inequality), to the Gini coefficient and Theil's entropy measures of inequality, are provided. The results suggest that (i) statistical inference is essential even when large samples are available, and (ii) the bootstrap appears to perform well in this setting.

Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests: An anova approach with dependent observations

Abstract: A method is proposed for comparing the accuracy of diagnostic tests that rely on a reader's subjective interpretation of the results. An ANOVA approach is applied where the dependencies between the readers' estimates of diagnostic accuracy are handled by adjusting the usual F statistic for theestimated correlation. The distribution of the resulting test statistic is evaluated. The problem is particularly relevant to diagnostic radiology where multi-reader studies of the indices associated with the ROC curve serve important roles in evaluating the efficacy of diagnostic tests.

Optimal experimental design and sample size for the statistical evaluation of data from somatic mutation and recombination tests (SMART) in Drosophila

Abstract: In genetic toxicology it is important to know whether chemicals should be regarded as clearly hazardous or whether they can be considered sufficiently safe, which latter would be the case from the genotoxicologist's view if their genotoxic effects are nil or at least significantly below a predefined minimal effect level. A previously presented statistical decision procedure which allows one to make precisely this distinction is now extended to the question of how optimal experimental sample size can be determined in advance for genotoxicity experiments using the somatic mutation and recombination tests (SMART) of Drosophila. Optimally, the statistical tests should have high power to minimise the chance for statistically inconclusive results. Based on the normal test, the statistical principles are explained, and in an application to the wing spot assay, it is shown how the practitioner can proceed to optimise sample size to achieve numerically satisfactory conditions for statistical testing. The somatic genotoxicity assays of Drosophila are in principle based on somatic spots (mutant clones) that are recovered in variable numbers on individual flies. The underlying frequency distributions are expected to be of the Poisson type. However, some care seems indicated with respect to this latter assumption, because pooling of data over individuals, sexes, and experiments, for example, can (but need not) lead to data which are overdispersed, i.e, the data may show more variability than theoretically expected. It is an undesired effect of overdispersion that in comparisons of pooled totals it can lead to statistical testing which is too liberal, because overall it yields too many seemingly significant results. If individual variability considered alone is not contradiction with Poisson expectation, however, experimental planning can help to minimise the undesired effects of overdispersion on statistical testing of pooled totals. The rule for the practice is to avoid disproportionate sampling. It is recalled that for optimal power in statistical testing, it is preferable to use equal total numbers of flies in the control and treated series. Statistical tests which are based on Poisson expectations are too liberal if there is overdispersion in the data due to excess individual variability. In this case we propose to use the U test as a non-parametric two-sample test and to adjust the estimated optimal sample size according to (i) the overdispersion observed in a large historical control and (ii) the relative efficiency of the U test in comparison to the t test and related parametric tests.

Biostatistics: Experimental Design and Statistical Inference.

Abstract: Introduction SECTION 1: EXPERIMENTAL DESIGN 1. The anatomy of an experiment 2. The anatomy of a scientific paper 3. Evaluation of a scientific article 4. Experimental Design SECTION 2: STATISTICAL INFERENCE, ANALYSIS OF VARIANCE AND POST-HOC 5. Statistical inference 6. Analysis of variance (ANOVA) 7. Post-hoc analysis SECTION 3: STATISTICAL TESTS 8. Parametric tests 9. Special ANOVA designs and analyses 10. Popular post-hoc multiple comparison tests 11. Nonparametric tests 12. Selection of statistical tests SECTION 4: RESEARCH DESIGN PROBLEMS AND THEIR CRITIQUES Research Design Problem: Sample Topic Index of Research Design Problems Postscript Appendix References

Interior-branch and bootstrap tests of phylogenetic trees.

Abstract: We have compared statistical properties of the interior-branch and bootstrap tests of phylogenetic trees when the neighbor-joining tree-building method is used. For each interior branch of a predetermined topology, the interior-branch and bootstrap tests provide the confidence values, PC and PB, respectively, that indicate the extent of statistical support of the sequence cluster generated by the branch. In phylogenetic analysis these two values are often interpreted in the same way, and if PC and PB are high (say, > or = 0.95), the sequence cluster is regarded as reliable. We have shown that PC is in fact the complement of the P-value used in the standard statistical test, but PB is not. Actually, the bootstrap test usually underestimates the extent of statistical support of species clusters. The relationship between the confidence values obtained by the two tests varies with both the topology and expected branch lengths of the true (model) tree. The most conspicuous difference between PC and PB is observed when the true tree is starlike, and there is a tendency for the difference to increase as the number of sequences in the tree increases. The reason for this is that the bootstrap test tends to become progressively more conservative as the number of sequences in the tree increases. Unlike the bootstrap, the interior-branch test has the same statistical properties irrespective of the number of sequences used when a predetermined tree is considered. Therefore, the interior-branch test appears to be preferable to the bootstrap test as long as unbiased estimators of evolutionary distances are used. However, when the interior-branch is applied to a tree estimated from a given data set, PC may give an overestimate of statistical confidence. For this case, we developed a method for computing a modified version (P'C) of the PC value and showed that this P'C tends to give a conservative estimate of statistical confidence, though it is not as conservative as PB. In this paper we have introduced a model in which evolutionary distances between sequences follow a multivariate normal distribution. This model allowed us to study the relationships between the two tests analytically.

Detection of gross erros in data reconciliation by principal component analysis

Abstract: Statistical testing prouides a tool for engineers and operators to judge the validity of process measurements and data reconciliation. Uniuariate, maximum power and chisquare tests haue been widely used for this purpose. Their perJormance, however, has not always been satisfactory. A new class of test statistics for detection and identification of gross errors is presented based on principal component analysis and is compared to the other statistics. It is shown that the new test is capable of detecting gross errors of small magnitudes and has substantial power to correctly identify the uariables in error, when the other tests fail.