The rapid development of the theory of robust estimation (Huber, 1973) has created a need for computational procedures to produce robust estimates. We will review a number of different computational approaches for robust linear regression but focus on one—iteratively reweighted least-squares (IRLS). The weight functions that we discuss are a part of a semi-portable subroutine library called ROSEPACK (RObust Statistical Estimation PACKage) that has been developed by the authors and Virginia Klema at the Computer Research Center of the National Bureau of Economic Research, Inc. in Cambridge, Mass. with the support of the National Science Foundation. This library (Klema, 1976) makes it relatively simple to implement an IRLS regression package.

Robust regression using iteratively reweighted least-squares

A number of methods have been proposed for dealing with extra-Poisson variation when doing regression analysis of count data. This paper studies negative-binomial regression models and examines efficiency and robustness properties of inference procedures based on them. The methods are compared with quasilikelihood methods.



Plusieurs methodes ont ete proposees en vue de traiter le probleme de la variation extra-poissonnienne dans une analyse de regression pour donnees de denombrement. Cet article a pour objet l'etude de modeles de regression binomiale negative et se penche sur les proprietes d'efficacite et de robustesse des methodes inferentielles decoulant des modeles. Ces dernieres sont comparees aux methodes de quasi-vraisemblancce.

/pdf/negative-binomial-and-mixed-poisson-regression-3p76ncbc6y.pdf

Negative binomial and mixed Poisson regression

Pseudo maximum likelihood techniques are applied to basic Poisson models and to Poisson models with specification errors. In the latter case it is shown that consistent and asymptotically normal estimators can be obtained without specifying the p.d.f. of the disturbances. These estimators are compared both from the finite sample and the asymptotic point of view. Quasi generalized PML estimators, which asymptotically dominate all PML estimators, are also proposed. Finally, bivariate and panel data Poisson models are discussed. THE ANALYSIS OF ECONOMIC BEHAVIOR often leads to the study of characteristics taking a small number of positive values. The classical linear model is not adapted to explain how such discrete variables depend on other quantitative or qualitative variables. The reasons are similar to those usually given in the case of an endogenous qualitative variable: the shape of the observation set does not correspond to a linear model, the assumption of normality of the disturbances cannot be made, since the endogenous variables take a small number of values with strictly positive probabilities, and the prediction formulae which are deduced from a linear model give impossible values. In the models considered in the literature to describe discrete variables (Cox and Lewis [2], El. Sayyad [3], Frome, Kutner, and Beauchamp [4], Gilbert [5], Hausman, Hall, and Griliches [8]; see also Lancaster [12]) the endogenous variable is assumed to have a Poisson distribution conditional upon the exogenous variables. The parameter of this distribution is a function of the values of the exogenous variables. The choice of such a model is justified if the dependent variable counts the occurrence of a given event during a fixed period and if the usual assumptions of the Poisson process are satisfied. For instance, the model is adapted to describe daily numbers of oil tankers' arrivals in a port, the number of accidents at work by factory, or the number of patents applied for and received by firms (Hausman, Hall, and Griliches [8]).

Pseudo Maximum Likelihood Methods: Applications to Poisson Models

We present several modifications of the Poisson and negative binomial models for count data to accommodate cases in which the number of zeros in the data exceed what would typically be predicted by either model. The excess zeros can masquerade as overdispersion. We present a new test procedure for distinguishing between zero inflation and overdispersion. We also develop a model for sample selection which is analogous to the Heckman style specification for continuous choice models. An application is presented to a data set on consumer loan behavior in which both of these phenomena are clearly present.

Accounting for Excess Zeros and Sample Selection in Poisson and Negative Binomial Regression Models

Models are considered in which the underlying rate at which events occur can be represented by a regression function that describes the relation between the predictor variables and the unknown parameters. Estimates of the parameters can be obtained by means of iteratively reweighted least squares (IRLS). When the events of interest follow the Poisson distribution, the IRLS algorithm is equivalent to using the method of scoring to obtain maximum likelihood (ML) estimates. The general Poisson regression models include log-linear, quasilinear and intrinsically nonlinear models. The approach considered enables one to concentrate on describing the relation between the dependent variable and the predictor variables through the regression model. Standard statistical packages that support IRLS can then be used to obtain ML estimates, their asymptotic covariance matrix, and diagnostic measures that can be used to aid the analyst in detecting outlying responses and extreme points in the model space. Applications of these methods to epidemiologic follow-up studies with the data organized into a life-table type of format are discussed. The method is illustrated by using a nonlinear model, derived from the multistage theory of carcinogenesis, to analyze lung cancer death rates among British physicians who were regular cigarette smokers.

The Analysis of Rates Using Poisson Regression Models

The method of iterative weighted least squares can be used to estimate the parameters in a nonlinear regression model. If the dependent variables are observations from a member of the regular exponential family, then under mild conditions it is shown that the IWLS estimates are identical to those obtained using the maximum likelihood principle. An application is provided to illustrate the results.

The equivalence of generalized least squares and maximum likelihood estimates in the exponential family.

The principle of maximum likelihood is used to obtain estimates of the parameters in a regression model when the experimental observations are assumed to follow the Poisson distribution. The maximum likelihood estimates are shown to be equivalent to those obtained by minimization of a quadratic form which reduces to a modified chi square under the Poisson assumption. Computationally, both of these estimation procedures are equivalent to a properly weighted least squares analysis. Approximate tests of the assumed Poisson variation and “goodness of fit” of the data to the model are proposed. Applications of the estimation procedure to linear and nonlinear regression models are discussed, and numerical examples are presented.

Regression Analysis of Poisson-Distributed Data

A comparison is made between linear programming and weighted least-squares approaches to absolute deviation curve fitting The computational results show that linear programming substantially reduces solution times while requiring only a relatively small amount of additional storage

A Comparison of Two Algorithms for Absolute Deviation Curve Fitting

SUMMARY A maximum likelihood procedure is presented for the estimation of the parameters in a survival curve which is used in the quantitative investigation of cytological damage resulting from ionizing radiation. This estimation procedure is developed under the assumption that the observations are distributed as independent Poisson random variables. In addition, a weighted least squares procedure, which gives estimates equivalent to the maximum likelihood estimates, is presented. Tests of the model and of the assumed distribution of the observations are given. Two illustrative examples are included.

Maximum likelihood estimation of survival curve parameters.

Hydroflumethiazide plasma and urine levels were measured fluorometrically following the oral administration of 100-mg doses (2 X 50 mg tablets) to each of 12 healthy, adult male subjects. The plasma level data as a function of time were fit to a one-compartment open model using a nonlinear regression digital computer program. The means of the individual apparent absorption and disposition rate apparent absorption and disposition rate constants were 0.770+/-0.46 hr(-1) (t1/2=0.9 hr) and 0.358+/-0.17 hr(-1) (t1/2=1.94 hr), respectively, with a 0.49+/-0.26 hr lag time in absorption. The same parameter estimates obtained by fitting the mean plasma concentration-time curve were 0.461+/-0.15 hr(-1), 0.357+/-0.13 hr(-1), and 0.39+/-0.09 hr for the apparent absorption and disposition rate constants and the lag time, respectively. The closeness of the parameter estimates and the lack of intravenous data preclude unequivocal assignment of the rate constants to the absorption or disposition processes. Nevertheless, the disposition of hydroflumethiazide is clearly a rapid process. The means of the individual peak plasma levels and times to peak were 0.301+/-0.94 microgram/ml and 2.12+/-0.59 hours, respectively. The mean area under the plasma level-time curve was 1.95+/-0.78 microgram-hr/ml and 46.7+/-20.0% of the administered dose was recovered in the 24-hour urine. The mean renal clearance of hydroflumethiazide was 453.4+/-256.6 ml/min.

Edward L. Frome

Papers

The equivalence of generalized least squares and maximum likelihood estimates in the exponential family.

Regression Analysis of Poisson-Distributed Data

A Comparison of Two Algorithms for Absolute Deviation Curve Fitting

Maximum likelihood estimation of survival curve parameters.

Pharmacokinetics of Orally Administered Hydroflumethiazide in Man