Showing papers on "Statistical hypothesis testing published in 1989"

The Great Crash, the Oil Price Shock, and the Unit Root Hypothesis

Abstract: We consider the null hypothesis that a time series has a unit root with possibly nonzero drift against the alternative that the process is «trend-stationary». The interest is that we allow under both the null and alternative hypotheses for the presence for a one-time change in the level or in the slope of the trend function. We show how standard tests of the unit root hypothesis against trend stationary alternatives cannot reject the unit root hypothesis if the true data generating mechanism is that of stationary fluctuations around a trend function which contains a one-time break

Comparisons of Treatments After an Analysis of Variance in Ecology

Abstract: The statistical literature on tests to compare treatments after the analysis of variance is reviewed, and the use of these tests in ecology is examined. Monte Carlo simulations on normal and lognormal data indicate that many of the tests commonly used are inappropriate or inefficient. Particular tests are recommended for unplanned multiple comparisons on the basis of controlling experimentwise type I error rate and providing maximum power. These include tests for parametric and nonparametric cases, equal and unequal sample sizes, homogeneous and heterogeneous variances, non-independent means (repeated measures or adjusted means), and comparing treatments to a control. Formulae and a worked example are provided. The problem of violations of assumptions, especially variance heterogeneity, was investigated using simulations, and particular strategies are recommended. The advantages and use of planned comparisons in ecology are discussed, and the philosophy of hypothesis testing with unplanned multiple comparisons is consid- ered in relation to confidence intervals and statistical estimation.

The statistical analysis of mitochondrial DNA polymorphisms: chi 2 and the problem of small samples.

Abstract: Significance levels obtained from a x2 contingency test are suspect when sample sizes are small. Traditionally this has meant that data must be combined. However, such an approach may obscure heterogeneity and hence potentially reduce the power of the statistical test. In this paper, we present a Monte Carlo solution to this problem: by this method, no lumping of data is required, and the accuracy of the estimate of c1 (i.e., a type 1 error) depends only on the number of randomizations of the original data set. We illustrate this technique with data from mtDNA studies, where numerous genotypes are often observed and sample sizes are relatively small.

Advanced BASIC meta-analysis

Abstract: Contents: Preface. Meta-Analysis: An Introduction. Defining the Hypothesis Test. Retrieving the Studies. Retrieving Statistical Tests of the Hypothesis. Retrieving Predictors of Study Outcomes. Combinations of Significance Levels and Effect Sizes. Diffuse Comparisons of Significance Levels and Effect Sizes. Focused Comparisons of Significance Levels and Effect Sizes (Complex Effects). Focused Comparisons of Significance Levels and Effect Sizes (Simple Effects). The Most Commonly Asked Questions About Meta-Analysis. Advanced BASIC Meta-Analysis: An Integrated Database Management System and Statistical Package.

The problem of statistical power in MIS research

Abstract: Statistical power is a topic of importance to any researcher using statistical inference testing. Studies with low levels of statistical power usually result in inconclusive findings, even though the researcher may have expended much time and effort gathering the data for analysis. A survey of the statistical power of articles employing statistical inference testing published in leading MIS journals shows that their statistical power is, on average, substantially below accepted norms. The consequence of this low power is that MIS researchers typically have a 40 percent chance of not detecting the phenomenon under study, even though it, in fact, may exist.Fortunately, there are several techniques, beyond expanding the sample size (which often may be impossible) that researchers can use to improve the power of their studies. Some are as easy as using a different but more powerful statistical test, while others require developing more elaborate sampling plans or a more careful construction of the research design. Attention tot he statistical power of a study is one key ingredient in assuring the success of the study. This article should serve as a useful guide for MIS researcher sin the planning, execution, and interpretation of inferential statistical analyses.

Generalized Monte Carlo significance tests

Abstract: SUMMARY Simple Monte Carlo significance testing has many applications, particularly in the preliminary analysis of spatial data. The method requires the value of the test statistic to be ranked among a random sample of values generated according to the null hypothesis. However, there are situations in which a sample of values can only be conveniently generated using a Markov chain, initiated by the observed data, so that independence is violated. This paper describes two methods that overcome the problem of dependence and allow exact tests to be carried out. The methods are applied to the Rasch model, to the finite lattice Ising model and to the testing of association between spatial processes. Power is discussed in a simple case.

Methods for the Comparative Analysis of Variation Patterns

Abstract: -Although comparisons of variation patterns with theoretical expectations and across species are playing an increasingly important role in systematics, there has been a lack of appropriate procedures for statistically testing the proposed hypotheses. We present a series of statistical tests for hypotheses of morphological integration and for interspecific comparison, along with examples of their application. These tests are based on various randomization and resampling procedures, such as Mantel's test with its recent extensions and bootstrapping. They have the advantage of avoiding the specific and strict distributional assumptions invoked by analytically-based statistics. The statistical procedures described include one for testing the fit of observed correlation matrices to hypotheses of morphological integration and a related test for significant differences in the fit of two alternative hypotheses of morphological integration to the observed correlation structure. Tests for significant similarity in the patterns and magnitudes of variance and correlation among species are also provided. [Morphometrics; comparative analysis; morphological integration; quadratic assignment procedures; Mantel's test; bootstrap.] Comparing observed patterns of morphometric variation to theories of morphological integration (Olson and Miller, 1958; Cheverud, 1982) and among species, or subspecific populations (Arnold, 1981; Riska, 1985), has been a largely ad hoc procedure. Previously, a large body of methods has been used to analyze variation patterns, including various forms of cluster analysis, factor analysis, principal components, multi-dimensional scaling, matrix correlations, and visual inspection. The results of such analyses were then discussed relative to some theory of variation patterns or compared between species or populations. These comparisons might either be verbal or quantitative, but tests of statistical significance were rarely employed. More recently, there has been an increase in statistical rigor in the field, particularly involving the use of quadratic assignment procedures (QAP; sometimes referred to as Mantel's test) (Mantel, 1967; Deitz, 1983; Dow and Cheverud, 1985; Smouse et al., 1986; Dow et al., 1987a, b; Hubert, 1987) for testing the statistical significance of matrix comparisons (Cheverud and Leamy, 1985; Lofsvold, 1986; Kohn and Atchley, 1988; Cheverud, 1989a; Wagner, 1989) and the use of confirmatory factor analysis (Zelditch, 1987, 1988) for testing hypotheses concerning levels and patterns of variation. These new methods allow statistical inference for hypotheses of morphological integration and for comparisons across species. We will describe the use of several of these newer methods, especially those using randomization, for testing hypotheses of morphological integration and interspecific comparison and provide brief examples of their use. The procedures described below can be used to rigorously test hypotheses concerning the causes of morphological variation and covariation patterns. A closely related set of procedures can be directed towards comparative, cross-taxon, analyses of variation and correlation patterns. The systematic study of distinction among group means is well known and extensively represented in the literature. However, systematic studies of variation patterns (as measured by a multivariate variance/covariance or correlation matrix) have been relatively rare. This has been due, in part, to a lack of relevant theory and appropriate systematic methodology. Important theoretical advances over the

Choice of structural model via parsimony: a rationale based on precision

Abstract: It is shown that, in large samples, the more parsimonious of two competing nested models yields an estimator of the common parameters that has smaller sampling variance. The use of parsimony as a criterion for choice between two otherwise acceptable models can thus be rationalized on the basis of precision of estimation.

An additional hidden unit test for neglected nonlinearity in multilayer feedforward networks

Abstract: The author presents a statistical test of the hypothesis that a given multilayer feedforward network exactly represents some unknown mapping subject to inherent noise against the alternative that the network neglects some nonlinear structure in the mapping, leading to potentially avoidable approximation errors. The tests are based on methods that statistically determine whether or not there is some advantage to be gained by adding hidden units to the network. >

Statistical Inference in Instrumental Variables

Abstract: This paper studies the asymptotic properties of instrumental variable (IV) estimates of multivariate cointegrating regressions. The framework of study is based on earlier work by Phillips and Durlauf (1986) and Park and Phillips (1988, 1989). In particular, the results in these papers are extended to allow for IV regressions that accommodate deterministic and stochastic regressors as well as quite general deterministic processes in the data generating mechanism. It is found that IV regressions are consistent even when the instruments are stochastically independent of the regressors. This phenomenon, which contrasts with traditional theory for stationary time series, is a beneficial artifact of spurious regression theory whereby stochastic trends in the instruments ensure their relevance asymptotically. Problems of inference are also addressed and some promising new theoretical results are reported. These involve a class of Wald tests which are modified by semiparametric corrections for serial correlation and for endogeneity. The resulting test statistics which we term fully modified Wald tests have limiting chi-squared distributions, thereby removing the obstacles to inference in cointegrated systems that were presented by the nuisance parameter dependencies in earlier work. Interestingly, IV methods themselves are insufficient to achieve this end and an endogeneity correction is still generally required, again in contrast to traditional theory. Our results therefore provide strong support for the conclusion reached by Hendry (1986) that there is no free lunch in estimating cointegrated systems. Some simulation results are reported which seek to explore the sampling behavior of our suggested procedures. These simulations compare our fully modified (semiparametric) methods with the parametric error correction methodology that has been extensively used in recent empirical research and with conventional least squares regression. Both the fully modified and error correction methods work well in finite samples and the sampling performance of each procedure confirms the relevance of asymptotic distribution theory, as distinct from superconsistency results, in discriminating between different statistical methods.

Alternative Test Criteria in Covariance Structure Analysis: A Unified Approach.

Abstract: In the context of covariance structure analysis, a unified approach to the asymptotic theory of alternative test criteria for testing parametric restrictions is provided. The discussion develops within a general framework that distinguishes whether or not the fitting function is asymptotically optimal, and allows the null and alternative hypothesis to be only approximations of the true model. Also, the equivalent of the information matrix, and the asymptotic covariance matrix of the vector of summary statistics, are allowed to be singular. When the fitting function is not asymptotically optimal, test statistics which have asymptotically a chi-square distribution are developed as a natural generalization of more classical ones. Issues relevant for power analysis, and the asymptotic theory of a testing related statistic, are also investigated.

From on-line to batch learning

Abstract: We contrast on-line and batch settings for concept learning, and describe an on-line learning model in which no probabilistic assumptions are made. We briefly mention some of our recent results pertaining to on-line learning algorithms developed using this model. We then turn to the main topic, which is an analysis of a conversion to improve the performance of on-line learning algorithms in a batch setting. For the batch setting we use the PAC-learning model. The conversion is straightforward, consisting of running the given on-line algorithm, collecting the hypotheses it uses for making predictions, and then choosing the hypothesis among them that does the best in a subsequent hypothesis testing phase. We have developed an analysis, using a version of Chernoff bounds applied to supermartingales, that shows that for some target classes the converted algorithm will be asymptotically optimal.

A maximum likelihood framework for determining moving edges

Abstract: The determination of moving edges in an image sequence is discussed An approach is proposed that relies on modeling principles and likely hypothesis testing techniques A spatiotemporal edge in an image sequence is modeled as a surface patch in a 3-D spatiotemporal space A likelihood ratio test enables its detection as well as simultaneous estimation of its related attributes It is shown that the computation of this test leads to convolving the image sequence with a set of predetermined masks The emphasis is on a restricted but widely relevant and useful case of surface patch, namely the planar one In addition, an implementation of the procedure whose computation cost is merely equivalent to a spatial gradient operator is presented This method can be of interest for motion-analysis schemes, not only for supplying spatiotemporal segmentation, but also for extracting local motion information Moreover, it can cope with occlusion contours and important displacement magnitude Experiments have been carried out with both synthetic and real images >

Hypothesis Testing in Semiparametric and Nonparametric Models for Econometric Time Series

Abstract: A restriction on a semiparametric or nonparametric econometric time series model determines the value of a finite-dimensional functional τ of an infinite-dimensional nuisance function. The estimate of τ and its estimated covariance matrix use nonparametric probability and spectral density estimation. A consequent test of the restriction is given approximate large sample justification under absolute regularity on the time series and other conditions. The methodology relates closely to recent proposals of Powell, Stock, Stoker and Robinson in cross-sectional applications, but serial dependence generally affects the test statistic's form, as well as statistical theory.

Nonnormalities and Tests of Asset Pricing Theories

Abstract: The robustness of the multivariate test of Gibbons, Ross, and Shanken (1986) to nonnormalities in the residual covariance matrix is examined After considering the relative performance of various tests of normality, simulation techniques are used to determine the effects of nonnormalities on the multivariate test It is found that, where the sample nonnormalities are severe, the size and/or power of the test can be seriously misstated However, it is also shown that these extreme sample values may overestimate the population parameters Hence, we conclude that the multivariate test is reasonably robust with respect to typical levels of nonnormality IN TRADITIONAL HYPOTHESIS TESTING, a nonrandom test maps the values of a random variable into a sample space dichotomized into regions where a hypothesis is either accepted or rejected There are three possible outcomes from this process: (1) a correct decision, (2) a false rejection (Type I error), or (3) failure to reject the hypothesis when it is false (Type II error) Of the latter two types of error, an error of the first kind is usually considered less desirable So typically, a level of significance is selected with low probability of Type I error (eg, 005 or 001), and a test is chosen so as to maximize power (minimize probability of Type II error) for the specified level of Type I error' Knowledge of the relative level of these two errors is critical in assimilating the results of an experiment For example, if the power of a test is equal to its significance level (ie, a weak test), rejection of the null contains zero information2 Additionally, in constructing parametric tests of hypotheses, it is necessary to assume some distribution for the underlying data Consequently, when using parametric tests, rejection of the null is only equivalent to rejection of at least one of the underlying hypotheses (ie, the null hypothesis or the distributional assumption) Interestingly, the size (significance level) and power of procedures used to test

Fitting genetic models with LISREL: hypothesis testing.

Abstract: A brief introduction to the mathematical theory involved in model fitting is provided. The properties of maximum-likelihood estimates are described, and their advantages in fitting structural models are given. Identification of models is considered. Standard errors of parameter estimates are compared with the use of likelihood-ratio (L-R) statistics. For structural modeling, L-R tests are invariant to parameter transformation and give robust tests of significance. Some guidelines for fitting models to data collected from twins are given, with discussion of the relative merits of parsimony and data description.

Temporal variation in allele frequencies: testing the right hypothesis

Abstract: Although standard statistical tests (such as contingency chi-square or G tests) are not well suited to the analysis of temporal changes in allele frequencies, they continue to be used routinely in this context. Because the null hypothesis stipulated by the test is violated if samples are temporally spaced, the true probability of a significant test statistic will not equal the nominal α level, and conclusions drawn on the basis of such tests can be misleading. A generalized method, applicable to a wide variety of organisms and sampling schemes, is developed here to estimate the probability of a significant test statistic if the only forces acting on allele frequencies are stochastic ones (i.e., sampling error and genetic drift). Results from analyses and simulations indicate that the rate at which this probability increases with time is determined primarily by the ratio of sample size to effective population size. Because this ratio differs considerably among species, the seriousness of the error in using the standard test will also differ. Bias is particularly strong in cases in which a high percentage of the total population can be sampled (for example, endangered species). The model used here is also applicable to the analysis of parent-offspring data and to comparisons of replicate samples from the same generation. A generalized test of the hypothesis that observed changes in allele frequency can be satisfactorily explained by drift follows directly from the model, and simulation results indicate that the true α level of this adjusted test is close to the nominal one under most conditions.

Pearsons-X2 and the loglikelihood ratio statistic-G2: a comparative review

Abstract: Summary The importance of developing useful and appropriate statistical methods for analyzing discrete multivariate data is apparent from the enormous amount of attention this subject has commanded in the literature over the last thirty years. Central to these discussions has been Pearson's X2 statistic and the loglikelihood ratio statistic G2. Our review seeks to consolidate this fragmented literature and develop a unifying theme for much of this research. The traditional X2 and G2 statistics are viewed as members of the power-divergence family of statistics, and are linked through a single real-valued parameter. The principal areas covered in this comparative survey are small-sample comparisons of X2 and G2 under both classical (fixed-cells) assumptions and sparseness assumptions, efficiency comparisons, and various modifications to the test statistics (including parameter estimation for ungrouped data, data-dependent and overlapping cell boundaries, serially dependent data, and smoothing). Finally some future areas for research are discussed.

Stock Market Forecastability and Volatility: A Statistical Appraisal

Abstract: This paper presents and implements statistical tests of stock market forecastability and volatility that are immune from the severe statistical problems of earlier tests Although the null hypothesis of strict market efficiency is rejected, the evidence against the hypothesis is not overwhelming That is, the data do not provide evidence of gross violations of the conventional valuation model

Long Swings in the Exchange Rate: Are they in the Data and Do Markets Know It?

Abstract: The value of the dollar appears to move in one direction for long periods of time. We develop a new statistical model of exchange rate dynamics as a sequence of stochastic, segmented time trends. The paper implements new techniques for parameter estimation and hypothesis testing for this framework. We reject the null hypothesis that exchange rates follow a random walk in favor of our model of long swings. Our model also generates better forecasts than a random walk. We conclude that persistent movement in the value of the dollar is a fact that calls for greater attention in the theory of exchange rate behavior. The model is a natural framework for assessing the importance of the "peso problem" for the dollar. It allows for the expectation of future exchange rates to be influenced by the probability of a change in regime. We nonetheless reject uncovered interest parity. The forward premium appears frequently to put too high a probability on a change in regime.

The Effect of Simulation Order on Level Accuracy and Power of Monte Carlo Tests

Abstract: We discuss the effect of simulation order on level accuracy and power of Monte Carlo tests, in a very general setting. Both parametric problems, with or without nuisance parameters, and nonparametric problems are treated by a single unifying argument. It is shown that if the level of a Monte Carlo test is known only nominally, not precisely, then the level error of a Monte Carlo test is an order of magnitude less than that of the corresponding asymptotic test. This result is available whenever the test statistic is asymptotically pivotal, even if the number of simulations is held fixed as the sample size n increases. It implies that Monte Carlo methods are a real alternative to asymptotic methods. We also show that, even if the number of simulations is held fixed, a Monte Carlo test is able to distinguish between the null hypothesis and alternative hypotheses distant n-'12 from the null.

Sample size determination for confidence intervals on the population mean and on the difference between two population means.

Abstract: Sample size determination is usually based on the premise that a hypothesis test is to be used. A confidence interval can sometimes serve better than a hypothesis test. In this paper a method is presented for sample size determination based on the premise that a confidence interval for a simple mean, or for the difference between two means, with normally distributed data is to be used. For this purpose, a concept of power relevant to confidence intervals is given. Some useful tables giving required sample size using this method are also presented.

Linear versus Nonlinear Macroeconomies: A Statistical Test

Abstract: A statistical test based on the estimated bispectrum is presented, which can distinguish between the linear stochastic dynamics widely used in macroeconomic models and alternative nonlinear dynamic mechanisms, including both nonlinear stochastic models and nonlinear deterministic (chaotic) models. The test is applied to an aggregate stock market index and to an aggregate industrial production index. In both cases, the test easily rejects the null hypothesis of a linear stochastic generating mechanism. This result strongly suggests that nonlinear dynamics (deterministic or stochastic) should be an important feature of any empirically plausible macroeconomic model. Copyright 1989 by Economics Department of the University of Pennsylvania and the Osaka University Institute of Social and Economic Research Association.

Testing for a Random Walk: A Simulation Experiment of Power When the Sampling Interval is Varied

Abstract: This paper analyzes the power of various tests for the random walk hypothesis against AR(1) alternatives when the sampling interval is allowed to vary. The null and alternative hypotheses are set in terms of the parameters of a continuous time model. The discrete time representations are derived and it is shown how they depend on the sampling interval. The power is simulated for a grid of values of the number of observations and the span of the data available (hence for various sampling intervals). Various test statistics are considered among the following classes: (a) test for a unit root on the original series and (b) tests for randomness in the differenced series. Among class (b), we consider both parametric and nonparametric tests, the latter including tests based on the rank of the first-differenced series. The paper therefore not only provides information as to the relative power of these tests but also about their properties when the sampling interval varies. This work is an extension of Perron (1987) and Shiller and Perron (1985).

estimation of spatial covariance structures by adjoint state maximum likelihood cross validation: 1. Theory

Abstract: This series of three papers describes a cross-validation method to estimate the spatial covariance structure of intrinsic or nonintrinsic random functions from point or spatially averaged data that may be corrupted by noise. Any number of relevant parameters, including nugget effect, can be estimated. The theory, described in this paper, is based on a maximum likelihood approach which treats the cross-validation errors as Gaussian. Various a posteriori statistical tests are used to verify this hypothesis and to show that in many cases, correlation between these errors is weak. The log likelihood criterion is optimized through a combination of conjugate gradient algorithms. An adjoint state theory is used to efficiently calculate the gradient of the estimation criterion, optimize the step size downgradient, and compute a lower bound for the covariance matrix of the estimation errors. Issues related to the identifiability, stability, and uniqueness of the resulting adjoint state maximum likelihood cross-validation (ASMLCV) method are discussed. This paper also describes the manner in which ASMLCV allows one to use model structure identification criteria to select the best covariance model among a given set of alternatives. Practical aspects of ASMLCV and its application to synthetic data are presented in paper 2 (Samper and Neuman, this issue (a)). Applications to real hydrogeological data (transmissivities and groundwater levels) have been presented elsewhere, while hydrochemical and isotopic data are analyzed by ASMLCV in paper 3 (Samper and Neuman, this issue (b)).

Power in econometric applications

Abstract: This paper is concerned with the use of power properties of tests in econometric applications. Inverse power functions are defined. These functions are designed to yield summary measures of power that facilitate the interpretation of test results in practice. Simple approximations are introduced for the inverse power functions of Wald, likelihood ratio, Lagrange multiplier, and Hausman tests. These approximations readily convey the general qualitative features of the power of a test. Examples are provided to illustrate their usefulness in interpreting test results. A COMMON PROBLEM faced in applied econometrics is that of interpreting the results of a hypothesis test when the test fails to reject the null hypothesis. Most practitioners realize that just because a test fails to reject a hypothesis one cannot claim to accept it. Nevertheless, it is common for this to be ignored, since the practitioner is often in a position where he would like the outcome of the test to provide useful inferences whether or not the test rejects. The purpose of this paper is to introduce inverse power (IP) summary measures that enable the practitioner to avoid such errors and make valid inferences when a test fails to reject the null hypothesis. These summary measures are widely applicable, easy to use (especially in the common case of a test concerning a single restriction), and simple to compute. When a test rejects the null hypothesis, the implication is that the data are inconsistent with each parameter point in the null in the sense that the probabil- ity of type I error for each point is small, viz., a or less. Correspondingly, when a test fails to reject the null hypothesis an analogous statement is needed regarding the error probabilities for points in the alternative hypothesis. It is not the case that all points in the alternative are inconsistent with the data in the sense that their probability of type II error is small (a or less). It is possible, however, to determine the region S in the alternative parameter space that is inconsistent with the data in this sense. The IP function introduced below evaluated at