Showing papers on "Linear model published in 1971"

Identification in parametric models

Abstract: A theory of identification is developed for a general stochastic model whose probability law is determined by a finite number of parameters. It is shown under weak regularity conditions that local identifiability of the unknown parameter vector is equivalent to nonsingularity of the information matrix. The use of "reduced-form" parameters to establish identifiability is also analyzed. The general results are applied to the familiar problem of determining whether the coefficients of a system of linear simultaneous equations are identifiable. THE IDENTIFICATION PROBLEM concerns the possibility of drawing inferences from observed samples to an underlying theoretical structure. An important part of econometric theory involves the derivation of conditions under which a given structure will be identifiable. The basic results for linear simultaneous equation systems under linear parameter constraints were given by Koopmans and Rubin [10] in 1950. Extensions to nonlinear systems and nonlinear constraints were made by Wald [15], Fisher [4, 5, 6], and others. A summary of these results can be found in Fisher's comprehensive study [7]. The identification problem has also been thoroughly analyzed in the context of the classical single-equation errors-in-variables model. The basic papers here are by Neyman [12] and Reiers0l [13]. Most of this previous work on the identification problem has emphasized the special features of the particular model being examined. This has tended to obscure the fact that the problem of structural identification is a very general one. It is not restricted to simultaneous-equation or errors-in-variables models. As Koopmans and Reiers0l [9] emphasize, the identification problem is "a general and fundamental problem arising, in many fields of inquiry, as a concomitant of the scientific procedure that postulates the existence of a structure." In their important paper Koopmans and Reiers0l define the basic characteristics of the general identification problem. In the present paper we shall, in the case of a general parametric model, derive some identifiability criteria. These criteria include the standard rank conditions for linear models as special cases. Our approach is based in part on the information matrix of classical mathematical statistics. Since this matrix is a measure of the amount of information about the unknown parameters available in the sample, it is not surprising that it should be related to identification. For lack of identification is simply the lack of sufficient information to distinguish between alternative structures. The following results make this relationship more precise.2

The Assumptions of the Linear Regression Model

Abstract: The paper is prompted by certain apparent deficiences both in the discussion of the regression model in instructional sources for geographers and in the actual empirical application of the model by geographical writers. In the first part of the paper the assumptions of the two regression models, the 'fixed X' and the 'random X', are outlined in detail, and the relative importance of each of the assumptions for the variety of purposes for which regression analysis may be employed is indicated. Where any of the critical assumptions of the model are seriously violated, variations on the basic model must be used and these are reviewed in the second half of the paper. THE rapid increase in the employment of mathematical models in planning has led R. J. Colenutt to discuss 'some of the problems and errors encountered in building linear models for prediction'.1 Colenutt rightly points out that the mathematical framework selected for such models 'places severe demands on the model builder because it is associated with a highly restrictive set of assumptions . . . and it is therefore imperative that, if simple linear models are to be used in planning, their limitations should be clearly understood'.2 These models have also been widely used in geography, for descriptive and inferential purposes as well as for prediction, and there is abundant evidence that, like their colleagues in planning, many geographers, when employing these models, have not ensured that their data satisfied the appropriate assumptions. Thus many researchers appear to have employed linear models either without verifying a sufficient number of assumptions or else after performing tests which are irrelevant because they relate to one or more assumptions not required by the model. Furthermore, many writers, reportipg geographical research, have completely omitted to indicate whether any of the assumptions have been satisfied. This last group is ambiguous, and it is clearly not possible, unless the values of the variables are published, to judge whether the correct set of assumptions has been tested or, indeed, to ascertain whether any such testing has been performed at all. This problem partially arises from certain shortcomings in material which has been published with the specific objective, at least inter alia, of instructing geographers on the use of quantitative techniques. All of these sources make either incomplete or inaccurate specifications of the assumptions underlying the application of linear models, although it is encouraging to note that there has been a considerable improvement in the quality of this literature in recent years. Thus, there were four books and two articles published in the early and mid-Ig60s which may be classified as belonging to this body of literature,3 yet, in five of these six sources, only one of the assumptions of the model is mentioned and, even

A note on the selection of data transformations

Abstract: SUMMARY Recently Box & Cox (1964) and Fraser (1967) have proposed likelihood functions as a basis for choosing a transformation of data to yield a simple linear model. Here a simple, exact, test of significance is proposed for the consistency of the data with a postulated transformation within a given family. Confidence sets can be derived from this test. The power of the test may be estimated and used to predict the sharpness of the inferences to be derived from such an analysis. The methods are illustrated with examples from the paper by Box & Cox (1964). Box & Cox (1964) considered the choice of a transformation among a parametric family of data transformations to yield a simple, normal, linear model. They investigated two approaches to this problem and derived a likelihood function and a posterior distribution for the parameters of the transformation. Draper & Cox (1969) have found approximations for the precision of the maximum likelihood estimate. Fraser (1967) derived a different likelihood function which yields quite different inferences from those of Box & Cox (1964) in extreme cases where the number of parameters is close to the number of observations. Likelihood methods require repeated computations using a number of transforms of the original data. This can be troublesome if there is a multiparameter family of transformations. A further defect of the likelihood methods is that confidence limits and tests based on them have only asymptotic validity; the number of parameters must be small compared with the number of observations. This will not be the case for small data sets, paired comparison experiments and extreme cases. In the present paper, a method is proposed which has three possible advantages over direct calculation of likelihoods. Its main disadvantage is that it does not lead to such a clear graphical summary of conclusions as is given by a plot of a likelihood. The advantages are: (i) an 'exact' test of significance is obtained from which 'exact' confidence limits can be calculated; (ii) the amount of calculation is reduced if only one or a few transforms are to be tested; (iii) the precision with which the transformation can be estimated is capable of theoretical calculation.

Least squares estimation in the regression model with autoregressive-moving average errors

Abstract: SUMMARY To treat the problem of correlated errors in regression, a model in which the errors follow a stationary autoregressive-moving average time series is suggested. Simultaneous least squares estimation of the regression and the time series parameters is discussed, and it is shown that asymptotically the estimates obtained in this manner possess normal distributions, whether or not the errors themselves are normally distributed. The estimates of the regression parameters are uncorrelated with those of the time series parameters; the former are distributed as if they had arisen from a certain transformed model with uncorrelated errors, while the latter have the same covariance matrix as those from a stationary series with no deterministic component. The estimate of variance is also asymptotically normal. A Monte Carlo sampling study indicates that these results can serve as a useful approximation for samples of moderate size.

Physical nearest-neighbour models and non-linear time-series

Abstract: A general class of spatial-temporal Markov processes is defined leading to the standard spatial equilibrium distribution for nearest-neighbour models on a multi-dimensional lattice. Physical properties are obtainable from the marginal spatial spectral function. However, only the simplest one-dimensional case corresponds to a linear model with a readily derived spectrum. Non-linear models corresponding to two- and three-dimensional lattices are presented in their simplest terms, and a preliminary discussion of approximate solutions is included.

The analysis of categorical data from mixed models

Abstract: This paper is concerned with contingency tables which are analogous to the well-known mixed model in analysis of variance. The corresponding experimental situation involves exposing each of n subjects to each of the d levels of a given factor and classifying the d responses into one of r categories. The resulting data are represented in an r X r X ... X r contingency table of d dimensions. The hypothesis of priincipal interest is equality of the one-dimensioinal marginal distributions. Alternatively, if the r categories may be quantitatively scaled, then attention is directed at the hypothesis of equality of the mean scores over the d first order marginals. Test statistics are developed in terms of minimum Neyman X2 or equivalently weighted least squares analysis of underlying linear models. As such, they bear a strong resemblance to the Hotelling T2 procedures used with continuous data in mixed models. Several numerical examples are given to illustrate the use of the various methods discussed.

Optimal non-linear estimation†

Abstract: For the non-linear estimation problem with non-linear plant and observation models, white gaussian excitations and continuous data, the state-vector a posteriori probabilities for prediction and smoothing are obtained via the 'partition theorem'. Moreover, for the special class of non-linear estimation problems with linear models excited by white gaussian noise, and with non-gaussian initial state, explicit results are obtained for the a posteriori probabilities, the optimal estimates and the corresponding error-covariance matrices for filtering, prediction and smoothing. In addition, for the latter problem, approximate but simpler expressions are obtained by using a gaussian sum approximation of the initial state-vector probability density. As a special case of the above results, optimal linear smoothing algorithms are obtained in a new form.

Comparison of Various Microwave Breakdown Prediction Models

Abstract: This paper is concerned with presenting a comparison of various microwave breakdown prediction models when applied to predicting breakdown of a practical antenna system located in a partially ionized environment. Solutions for the breakdown conditions are obtained numerically, allowing for the field and environment properties to have arbitrary spatial variation. The models are applied to predictions of breakdown for an S‐band slot radiating from a reentry vehicle, and the enviroment properties are taken from aerodynamic conditions existing along the descent trajectory. The three models compared consist of a linear model, a variational model, and a nonlinear model. The models are based on the electron‐continuity equation and allow for the effects of diffusion (including the transition from ambipolar to free diffusion), high‐temperature effects, and recombination, which, to our knowledge, have not been treated previously in a single model.

Optimal nonlinear estimation

Abstract: For the nonlinear estimation problem with nonlinear plant and observation models, white gaussian excitations and continuous data, the state-vector a-posteriori probabilities for prediction, and smoothing are obtained via the "partition theorem". Moreover, for the special class of nonlinear estimation problems with linear models excited by white gaussian noise, and with nongaussian initial state, explicit results are obtained for the a-posteriori probabilities, the optimal estimates, and the corresponding error-covariance matrices for filtering, prediction, and smoothing. In addition, for the latter problem, approximate but simpler expressions are obtained by using a gaussian sum approximation of the initial state-vector probability density. As a special case of the above results, optimal linear smoothing algorithms are obtained in a new form.

Significance tests based on residuals

Abstract: SUMMARY The known distribution of residuals from linear regression models may be used to construct exact tests of significance. New tests for the presence of one or more outliers are considered in detail. Applications of the theory to other tests are discussed. Exact results are worked out for the normal and exponential error distributions; formulae are given for other nonnormal cases. All statistical tests are based on some model specifying the form or structure of the response. Linear models are a large and important class of the models currently used. The statistical tests based on linear models fall generally into two categories: (i) tests within the model that are sensitive to departures from some hypothesis about the parameters of the model; and (ii) tests of the model that are sensitive to departures from the assumptions of the model regardless of the parameters within the model. Tests of the latter type are based on the normalized residual vector, or some function of it, which has a known marginal distribution independent of the parameters of the model. Tests within the model are made conditionally given this ancillary residual vector. However, the distinction between these tests, at least for normal models, exists more in theory thain in practice. Section 2 contains some preliminary definitions and results which lead, in ? 3, to the distribution of residuals in both normal and nonnormal cases. In ? 4 a class of significance tests based on the structure of the regression problem is proposed and in ? 5 it is shown that the common analysis of variance tests for normal models belong to this class. In ? 6 this theory is applied to the problem of testing for the presence of one or more outliers. Examples of the derived tests are given for normal and exponential cases. Finally, in ? 7 the relation to other tests for nonadditivity and nonnormality is discussed.

New Estimators of Disturbances in Regression Analysis

Abstract: In this article, an alternative v for the vector u of least-squares residuals in the linear model is derived. It is best in the class of all linear unbiased estimators' of u having a certain fixed covariance matrix chosen a priori. Under the normality assumption, the distribution of the Von Neumann Ratio based on v is independent of the regression vectors, so that v is particularly useful for testing on serial correlation of the disturbances. It is pointed out that the existing tests for serial correlation in economic time-series models might be improved by using v based on an appropriate covariance matrix; the Durbin-Watson upper-bound tables can be used for this purpose.

Analysis of correlated random effects: linear model with two random components

Abstract: Linear regression models with two random components, one of which is time correlated, are analysed from a Bayesian viewpoint. The problem of making inferences about the parameters when the time correlated component is stationary is discussed. Necessary modificatiorLs needed for nonstationary and explosive models are indicated. The analysis is illustrated by a numerical example showing that the time series component of the data can exert a strong influence in determining the posterior distribution of the regression coefficients. Finally, the case of several linear models is considered and a possible application to seasonal series is indicated. 1. IN-rRODUCTION In this paper, we analyze from a Bayesian viewpoint the linear regression model with

Generalized inverses of partitioned matrices and recalculation of least squares estimates for data or model changes

Abstract: In this paper, we develop formulae similar to those of Greville and Cline con necting g-inverses of A with the corresponding g-inverses of (A : a) where a is a column vector. We consider A", Aj, A?, A^ and A+ for this purpose. In Sections 2 and 3 two parallel sets of formulae are developed, one for computing (A : a)~ from A~ and the other for computing A~ from (A : a)~. In Sections 2 and 3 we consider only the Euclidean norms. In Section 4, the results of Sections 2 and 3 are generalized where we consider more general inner products. The results of Sections 2, 3 and 4 are useful in least squares computations. First let us consider a linear model F = X$ + e where D(e) = a2I. The results of Section 2 are useful when one looks for revised estimates of parameters when either an additional uncorrelated observation with unit variance is considered or an extra parameter is added to the linear model. The results of Section 3 are useful when one wishes to compute revised estimates of the remaining parameters if a superfluous parameter is dropped out from the original linear model. These are also useful in

New Criteria for Estimability for Linear Models

Abstract: A new criterion for determining the estimability of linear combinations of the parameters of a linear model is established. The result consists of evaluating the trace of a matrix and thus only one number must be checked to determine estimability. The sums of squares necessary to test hypotheses about estimable linear combinations are also derived. Finally, a stepwise computational procedure to compute generalized inverses and matrix products involving generalized inverses is presented. Using the theory and computational techniques, a computer program can be developed to provide a complete analysis of the linear model using generalized inverses.

Estimation of heritability from experiments with inbred and related individuals.

Abstract: SUMMARY Assuming an arbitrary amount of inbreeding and certain types of relationships among individuals in the parent population, it is shown how for nested and diallel mating designs the genetic and environmental variance components can be estimated from the mean squares in the usual analyses of variance. The systems of equations are given explicitly for the case of equal numbers of offspring per mating. A method for dealing with unequal numbers of offspring is outlined. In a previous paper (Hinkelmann [1969]) modifications of the usual procedures for estimating heritability from nested and diallel mating designs were given for the case that the dams are related. It was shown how this relationship induces a certain covariance structure for some random effects in the underlying linear model. However, just as the variance components can be expressed in terms of genetic and environmental variance components, these covariance components are simply functions of the genetic variance components and the coefficients of relationship. Since the coefficients of relationship can be evaluated from the pedigree, it is then easy to estimate the genetic and environmental variance components from the mean squares in the usual analysis of variance, and hence estimate heritability. In the present paper this method is extended to allow for an arbitrary amount of inbreeding and certain types of relationships in the parental population. The most general situation would be to allow for all types of relationships, i.e., among sires, among dams, and among sires and dams. This, however, may lead to inbreeding in the offspring generation which in turn leads to problems in evaluatiing covariances between relatives, an approach on which the results in this paper are based. Harris [1964] has developed formulae for the covariance between inbred relatives which involve additional genetic parameters not present in the usual covariance formulae for non-inbred relatives (e.g. Kempthorne [1957]). These parameters are not estimable from the usual mating designs. Also, Cockerham [1963] and Harris [19641 give special conditions under which the covariance between inbred

Best Linear Recursive Estimation

Abstract: This article presents a matrix formulation of recursive forms for best linear unbiased estimators [Xcirc]N of the parameter vector x in the linear model yi=hix+ei, i = 1,2, ···, N when the observation vectors yi are correlated. If data are collected in sequence one can formulate recursive forms of the estimator [Xcirc]N so that it is not necessary to store all the previous data but only previous estimates and current data. This requires less storage space to obtain best linear unbiased estimators. This is especially advantageous in real-time estimation problems.

lARGE SAMPLE VARIANCES OF MAXIMUM LIKELIHOOD ESTIMATORS OF VARIANCE COMPONENTS IN THE 3-WAY NESTED ClASSIFICATION, RANDOM MODEL, WITH UNBAlANCED DATA

Abstract: Summary Explicit expressions are presented for the elements of the information matrix of the variance components in a 3-way nested classification, random model, with normality and unbalanced data. 1. Introduction and Model ~ Searle [1970] developed a general method for obtaining, under normality conditions, the elements of the information matrix of the variance components of mixed models, with unbalanced data; in particular he displayed the results for the 2-way nested classification. This paper presents analogous results for the 3-way nested classification, random model, for the general case of unbalanced data. The linear model for an observation is taken to be

Mathematical Models for Aversive Conditioning

Abstract: Publisher Summary This chapter discusses some mathematical models for aversive conditioning, such as two-operator linear model, discrete performance-level Markov model, and general Markov model It focuses on a number of sets of response sequences from avoidance conditioning experiments that generated a hundred or more sequences to assess the commutativity and other sequential properties of the two-operator model It also presents data that indicate that the two-operator linear model is an inadequate approximation to active avoidance conditioning and choice reversal in a shock escape T-maze The general Markov model that requires discrete changes in performance has been shown to provide a reasonable description of the avoidance conditioning process in a large number of studies using rats as subjects Interpretation of the discrete performance model in terms of short-term and long-term memory processes has been proven fruitful because predictions that follow from the memory interpretation have been supported when experimental variables identified with the model's parameters were manipulated The discrete performance-level Markov model does not require stationarity of either the intermediate response probabilities or response latencies between the first avoidance and last error The chapter also reviews the development of these mathematical models for avoidance conditioning providing some preliminary terminological and notational considerations

Systematic computer-aided three-dimensional modeling of integrated bipolar devices

Abstract: A general, systematic method for modeling three-dimensional bulk effects in integrated bipolar devices is developed. The application of the technique to a device results in a hierarchy of large-signal models-based on linear differential equations-of variable accuracy and complexity from which the simplest but sufficiently accurate model of the device and its interactions with neighboring devices can be selected. The models produced consist of lumped elements which are directly related to the device morphology and to physical processes occurring in the device. The analogies existing between the terminal characteristics of the lumped elements and those of electrical elements are utilized in the formulation of a general lumped-to-electrical transformation technique by which equivalent electrical models, amenable to analysis by existing network analysis programs, can be obtained. The derived modeling theory is applied to several devices of variable complexity, and resulting models are verified experimentally. Finally, discussions concerning modifications of linear models to include, at least on first-order bases, several nonlinear effects, some associated with pn junctions, are given.