Showing papers in "Australian & New Zealand Journal of Statistics in 2004"

Partial correlation and conditional correlation as measures of conditional independence

Abstract: Summary This paper investigates the roles of partial correlation and conditional correlation as measures of the conditional independence of two random variables. It first establishes a sufficient condition for the coincidence of the partial correlation with the conditional correlation. The condition is satisfied not only for multivariate normal but also for elliptical, multivariate hypergeometric, multivariate negative hypergeometric, multinomial and Dirichlet distributions. Such families of distributions are characterized by a semigroup property as a parametric family of distributions. A necessary and sufficient condition for the coincidence of the partial covariance with the conditional covariance is also derived. However, a known family of multivariate distributions which satisfies this condition cannot be found, except for the multivariate normal. The paper also shows that conditional independence has no close ties with zero partial correlation except in the case of the multivariate normal distribution; it has rather close ties to the zero conditional correlation. It shows that the equivalence between zero conditional covariance and conditional independence for normal variables is retained by any monotone transformation of each variable. The results suggest that care must be taken when using such correlations as measures of conditional independence unless the joint distribution is known to be normal. Otherwise a new concept of conditional independence may need to be introduced in place of conditional independence through zero conditional correlation or other statistics.

Assessing congruenceamong distance matrices: single‐malt scotch whiskies revisited

Abstract: Summary A test of congruence among distance matrices is described. It tests the hypothesis that several matrices, containing different types of variables about the same objects, are congruent with one another, so they can be used jointly in statistical analysis. Raw data tables are turned into similarity or distance matrices prior to testing; they can then be compared to data that naturally come in the form of distance matrices. The proposed test can be seen as a generalization of the Mantel test of matrix correspondence to any number of distance matrices. This paper shows that the new test has the correct rate of Type I error and good power. Power increases as the number of objects and the number of congruent data matrices increase; power is higher when the total number of matrices in the study is smaller. To illustrate the method, the proposed test is used to test the hypothesis that matrices representing different types of organoleptic variables (colour, nose, body, palate and finish) in single-malt Scotch whiskies are congruent.

Prediction in linear mixed models

Abstract: Summary Following estimation of effects from a linear mixed model, it is often useful to form predicted values for certain factor/variate combinations. The process has been well defined for linear models, but the introduction of random effects into the model means that a decision has to be made about the inclusion or exclusion of random model terms from the predictions. This paper discusses the interpretation of predictions formed including or excluding random terms. Four datasets are used to illustrate circumstances where different prediction strategies may be appropriate: in an orthogonal design, an unbalanced nested structure, a model with cubic smoothing spline terms and for kriging after spatial analysis. The examples also show the need for different weighting schemes that recognize nesting and aliasing during prediction, and the necessity of being able to detect inestimable predictions.

Criteria for Linear Model Selection Based on Kullback's Symmetric Divergence

Abstract: Summary Model selection criteria are frequently developed by constructing estimators of discrepancy measures that assess the disparity between the 'true' model and a fitted approximating model. The Akaike information criterion (AIC) and its variants result from utilizing Kullback's directed divergence as the targeted discrepancy. The directed divergence is an asymmetric measure of separation between two statistical models, meaning that an alternative directed divergence can be obtained by reversing the roles of the two models in the definition of the measure. The sum of the two directed divergences is Kullback's symmetric divergence. In the framework of linear models, a comparison of the two directed divergences reveals an important distinction between the measures. When used to evaluate fitted approximating models that are improperly specified, the directed divergence which serves as the basis for AIC is more sensitive towards detecting overfitted models, whereas its counterpart is more sensitive towards detecting underfitted models. Since the symmetric divergence combines the information in both measures, it functions as a gauge of model disparity which is arguably more balanced than either of its individual components. With this motivation, the paper proposes a new class of criteria for linear model selection based on targeting the symmetric divergence. The criteria can be regarded as analogues of AIC and two of its variants: 'corrected' AIC or AICc and 'modified' AIC or MAIC. The paper examines the selection tendencies of the new criteria in a simulation study and the results indicate that they perform favourably when compared to their AIC analogues.

Estimating the first- and second-order parameters of a heavy-tailed distribution

Abstract: Summary This paper suggests censored maximum likelihood estimators for the first- and secondorder parameters of a heavy-tailed distribution by incorporating the second-order regular variation into the censored likelihood function. This approach is different from the biasreduced maximum likelihood method proposed by Feuerverger and Hall in 1999. The paper derives the joint asymptotic limit for the first- and second-order parameters under a weaker assumption. The paper also demonstrates through a simulation study that the suggested estimator for the first-order parameter is better than the estimator proposed by Feuerverger and Hall although these two estimators have the same asymptotic variances.

Switching Order Statistics through Random Power Contractions

Abstract: Summary This paper investigates a new random contraction scheme which complements the length-biasing and convolution contraction schemes considered in the literature. A random power contraction is used with order statistics, leading to new and elegant characterizations of the power distribution. In view of Rossberg's counter-example of a non-exponential law with exponentially distributed spacings of order statistics, possibly the most appealing consequence of the result is a characterization of the exponential distribution via an independent exponential shift of order statistics.

Asymptotics and Criticality for a Correlated Bernoulli Process

Abstract: Summary A generalized binomial distribution that was proposed by Drezner & Farnum in 1993 comes from a correlated Bernoulli process with interesting asymptotic properties which differ strikingly in the neighbourhood of a critical point. The basic asymptotics and a short-range/long-range dependence dichotomy are presented in this note.

Rescaling Marked Point Processes

Abstract: In 1971, Meyer showed how one could use the compensator to rescale a multivariate point process, forming independent Poisson processes with intensity one.

A randomized longitudinal play‐the‐winner design for repeated binary data

Abstract: Summary In some clinical trials with two treatment arms, the patients enter the study at different times and are then allocated to one of two treatment groups. It is important for ethical reasons that there is greater probability of allocating a patient to the group that has displayed more favourable responses up to the patient's entry time. There are many adaptive designs in the literature to meet this ethical constraint, but most have a single binary response. Often the binary response is longitudinal in nature, being observed repeatedly over different monitoring times. This paper develops a randomized longitudinal play-the-winner design for such binary responses which meets the ethical constraint. Some performance characteristics of this design have been studied. It has been implemented in a trial of pulsed electro-magnetic field therapy with rheumatoid arthritis patients.

Continuous‐Time Stochastic Processes with Cyclical Long‐Range Dependence

Abstract: We introduce continuous-time random processes whose spectral density is unbounded at some non-zero frequencies. The discretized versions of these processes have asymptotic properties similar to those of discrete-time Gegenbauer processes. We present some properties of the covariance function and spectral density as well as a theory of statistical estimation of the mean and covariance function of such processes. Some direction for further generalizations of the results are indicated.

Wilcoxon-type rank-sum precedence tests

Abstract: Summary This paper introduces Wilcoxon-type rank-sum precedence tests for testing the hypothesis that two life-time distribution functions are equal. They extend the precedence life-test first proposed by Nelson in 1963. The paper proposes three Wilcoxon-type rank-sum precedence test statistics—the minimal, maximal and expected rank-sum statistics—and derives their null distributions. Critical values are presented for some combinations of sample sizes, and the exact power function is derived under the Lehmann alternative. The paper examines the power properties of the Wilcoxon-type rank-sum precedence tests under a location-shift alternative through Monte Carlo simulations, and it compares the power of the precedence test, the maximal precedence test and Wilcoxon rank-sum test (based on complete samples). Two examples are presented for illustration.

A general inverse sampling scheme and its application to adaptive cluster sampling

Abstract: Summary Not having a variance estimator is a seriously weak point of a sampling design from a practical perspective. This paper provides unbiased variance estimators for several sampling designs based on inverse sampling, both with and without an adaptive component. It proposes a new design, which is called the general inverse sampling design, that avoids sampling an infeasibly large number of units. The paper provide estimators for this design as well as its adaptive modification. A simple artificial example is used to demonstrate the computations. The adaptive and non-adaptive designs are compared using simulations based on real data sets. The results indicate that, for appropriate populations, the adaptive version can have a substantial variance reduction compared with the non-adaptive version. Also, adaptive general inverse sampling with a limitation on the initial sample size has a greater variance reduction than without the limitation.

Ross‐Type Conjectures on Monotonicity of Queues

Abstract: In his seminal paper from 1978, Ross set up a few conjectures which formalize a common belief that more variable arrival processes lead to worse performance in queueing systems. We study these types of problems for Cox/GI/1/∞, Cox/GI/∞/∞, and Cox/GI/1/0 systems. Assumptions are stated in terms of ≤idcx-regularity. For example, in the class of stationary Markov processes, the regularity property holds under a doubly stochastic monotonicity assumption. A special case is a result of Daley (1968) on the decreasing covariance function for stochastically monotone stationary Markov processes.

Detection of a change point with local polynomial fits for the random design case

Abstract: Summary Regression functions may have a change or discontinuity point in the ν th derivative function at an unknown location. This paper considers a method of estimating the location and the jump size of the change point based on the local polynomial fits with one-sided kernels when the design points are random. It shows that the estimator of the location of the change point achieves the rate n −1/(2ν+1) when ν is even. On the other hand, when ν is odd, it converges faster than the rate n −1/(2ν+1) due to a property of one-sided kernels. Computer simulation demonstrates the improved performance of the method over the existing ones.

SHARP BOUNDS FOR EXPECTATIONS OF kTH RECORD INCREMENTS

Abstract: Summary This paper presents sharp bounds for expectations of non-adjacent increments of kth record statistics, measured in various scale units, for a sequence of independent identically distributed random variables with continuous cumulative distribution function. The results for kth record spacings are considered as special cases. The paper also characterizes probability distributions for which the bounds are attained.

Mid‐P confidence intervals based on the likelihood ratio for proportions estimated by group testing

Abstract: Summary Group testing has been used in many fields of study to estimate proportions. When groups are of different size, the derivation of exact confidence intervals is complicated by the lack of a unique ordering of the event space. An exact interval estimation method is described here, in which outcomes are ordered according to a likelihood ratio statistic. The method is compared with another exact method, in which outcomes are ordered by their associated MLE. Plots of the P-value against the proportion are useful in examining the properties of the methods. Coverage provided by the intervals is assessed using several realistic grouptesting procedures. The method based on the likelihood ratio, with a mid-P correction, is shown to give very good coverage in terms of closeness to the nominal level, and is recommended for this type of problem.

Fitting And Comparing Seed Germination Models With A Focus On The Inverse Normal Distribution

Abstract: Summary This paper reviews current methods for fitting a range of models to censored seed germination data and recommends adoption of a probability-based model for the time to germination. It shows that, provided the probability of a seed eventually germinating is not on the boundary, maximum likelihood estimates, their standard errors and the resultant deviances are identical whether only those seeds which have germinated are used or all seeds (including seeds ungerminated at the end of the experiment). The paper recommends analysis of deviance when exploring whether replicate data are consistent with a hypothesis that the underlying distributions are identical, and when assessing whether data from different treatments have underlying distributions with common parameters. The inverse normal distribution, otherwise known as the inverse Gaussian distribution, is discussed, as a natural distribution for the time to germination (including a parameter to measure the lag time to germination). The paper explores some of the properties of this distribution, evaluates the standard errors of the maximum likelihood estimates of the parameters and suggests an accurate approximation to the cumulative distribution function and the median time to germination. Additional material is on the web, at http://www.agric.usyd.edu.au/staff/oneill/.

Ultimate Extinction of the Promiscuous Bisexual Galton—Watson Metapopulation

Abstract: Summary A variant of a sexual Gallon–Watson process is considered. At each generation the population is partitioned among n‘hosts’ (population patches) and individual members mate at random only with others within the same host. This is appropriate for many macroparasite systems, and at low parasite loads it gives rise to a depressed rate of reproduction relative to an asexual system, due to the possibility that females are unmated. It is shown that stochasticity mitigates against this effect, so that for small initial populations the probability of ultimate extinction (the complement of an ‘epidemic’) displays a tradeoff as a function of n between the strength of fluctuations which overcome this ‘mating’ probability, and the probability of the subpopulation in one host being ‘rescued’ by that in another. Complementary approximations are developed for the extinction probability: an asymptotically exact approximation at large n, and for small n a short-time probability that is exact in the limit where the mean number of offspring per parent is large.

Issues Related to Sports Gambling

Abstract: Summary This paper looks at various issues that are of interest to the sports gambler. First, an expression is obtained for the distribution of the final bankroll using fixed wagers with a specified initial bankroll. Second, fixed percentage wagers are considered where the Kelly method is extended to the case of simultaneous bets placed at various odds; a computational algorithm is presented to obtain the Kelly fractions. Finally, the paper considers the problem of determining whether a gambling system is profitable based on the historical results of bets placed at various odds.

Prediction regions for bivariate extreme events

Abstract: This paper suggests using a mixture of parametric and non-parametric methods to construct prediction regions in bivariate extreme-value problems. The non-parametric part of the technique is used to estimate the dependence function, or copula, and the parametric part is employed to estimate the marginal distributions. A bootstrap calibration argument is suggested for reducing coverage error. This combined approach is compared with a more parametric one, relative to which it has the advantages of being more flexible and simpler to implement. It also enjoys these features relative to predictive likelihood methods. The paper shows how to construct both compact and semi-infinite bivariate prediction regions, and it treats the problem of predicting the value of one component conditional on the other. The methods are illustrated by application to Australian annual maximum temperature data. (Less)

Subexponential Distributions — Large Deviations with Applications to Insurance and Queueing Models

Abstract: Summary This paper presents a fine large-deviations theory for heavy-tailed distributions whose tails are heavier than exp(−√t and have finite second moment. Asymptotics for first passage times are derived. The results are applied to estimate the finite time ruin probabilities in insurance as well as the busy period in a GI/G/1 queueing model.

Buehler confidence limits and nesting

Abstract: Summary The Buehler 1 − α upper confidence limit is as small as possible, subject to the constraints that its coverage probability is at least 1 − α and that it is a non-decreasing function of a pre-specified statistic T. This confidence limit has important biostatistical and reliability applications. Previous research has examined the way the choice of T affects the efficiency of the Buehler 1 − α upper confidence limit for a given value of α. This paper considers how T should be chosen when the Buehler limit is to be computed for a range of values of α. If T is allowed to depend on α then the Buehler limit is not necessarily a non-increasing function of α, i.e. the limit is ‘non-nesting’. Furthermore, non-nesting occurs in standard and practical examples. Therefore, if the limit is to be computed for a range [αL ,α U ]o f values of α, this paper suggests that T should be a carefully chosen approximate 1 − αL upper limit for θ. The choice leads to Buehler limits that have high statistical efficiency and are nesting.

Statistical Analyses and Modelling of the Mixing Process of Euro Coins in Germany and Europe

Abstract: Summary This paper reports on the mixing of Euro coins from different countries of origin in Europe, which started on 1 January 2002. There is an interesting conclusion: that the mobility of small and large denominations is different. The long-term behaviour of the mixing process is studied using a simple deterministic model and data from Germany, France and the Netherlands. The analysis leads to predictions about the future progress of the mixing process.

Using Multinomial Mixture Models to Cluster Internet Traffic

Abstract: Summary The paper considers the clustering of two large sets of Internet traffic data consisting of information measured from headers of transmission control protocol packets collected on a busy arc of a university network connecting with the Internet. Packets are grouped into 'flows' thought to correspond to particular movements of information between one computer and another. The clustering is based on representing the flows as each sampled from one of a finite number of multinomial distributions and seeks to identify clusters of flows containing similar packet-length distributions. The clustering uses the EM algorithm, and the data-analytic and computational details are given.

Kernel Estimation of the Spectral Density of Stationary Random Closed Sets

Abstract: Anonparametrickernelestimatorofthespectraldensityofstationaryran-domclosedsetsisstudied. Conditionsarederivedunderwhichthisestimatorisasymptoticallyunbiasedandmean-squareconsistent. FortheplanarBooleanmodelwithisotropiccompactandconvexgrains,anaveragedversionoftheker-nelestimatoriscomparedwiththetheoreticalspectraldensity.Keywords Stationary random set; spectral density; nonparamet-ric kernel estimator; asymptotic unbiasdness; mean-square consis-tency; Boolean modelAMS2000subjectclassiflcation: 60D05;62G20 1 Introduction The problem of nonparametric estimation of the spectral density f(x) of stationaryrandom closed sets ¥ in IR d is studied, where ¥ is observed in a sampling windowW‰IR d withpositiveandflnitevolumejWj. Motivatedbyanapproximationformula,whichcanbeconcludedfromLemma2.2,thatisf(x)…1jWjIEflflflflZ IR d g(z)e ihx;z dzflflflfl 2 ; x2IR d ;whereg(z)= 1I W (z)(1I ¥ (z)ip)andp= IP(o2¥),thekernelestimatorfb n (x)=1jW n jb nd Z IR d kµxiyb n ¶flflflflZ W n (1I ¥ (z)ip)e ihy;z dzflflflfl 2

Corrected proof of the result of 'a prediction error property of the lasso estimator and its generalization' by huang (2003)

Abstract: Summary The Lasso achieves variance reduction and variable selection by solving an � 1-regularized least squares problem. Huang (2003) claims that ‘there always exists an interval of regularization parameter values such that the corresponding mean squared prediction error for the Lasso estimator is smaller than for the ordinary least square estimator’. This result is correct. However, its proof in Huang (2003) is not. This paper presents a corrected proof of the claim, which exposes and uses some interesting fundamental properties of the Lasso.

New Explicit Examples of Fixed Points of Poisson Shot Noise Transforms

Abstract: We show that gamma distributions, generalized positive Linnik distri-butions, S2 distributions are fixed points of Poisson shot noise transforms.The corresponding response functions are identified via their inverse func-tions except for some special cases when those can be obtained explicitly.As a by-product, it is proven that log-convexity of the response function isnot necessary for selfdecomposability of non-negative Poisson shot noisedistribution. Some attention is given to perpetuities of a rather specialtype which are closely related to our model. In particular, we study theproblem of their existence and uniqueness.Key words: Poisson shot noise transform · shot noise distribution ·fixed points · perpetuity · infinite divisibility · selfdecomposability 1 Introduction. 1 Let P + be the set of all probability distributions on the Borel subsets ofR + = [0,∞) and h : R + → R + be a Borel measurable function which in whatfollows we call response function . Fix a probability space (Ω,F,P). It will beassumed throughout the paper that all random variables (r.v.’s) involved aredefined there, and this space is rich enough to accumulate independent copiesof some r.v.’s. Also from now on notation µ = L(ξ) means that µ ∈ P

Ecological bias: use of maximum-entropy approximations

Abstract: Summary The focus of geographical studies in epidemiology has recently moved towards looking for effects of exposures based on data taken at local levels of aggregation (i.e. small areas). This paper investigates how regression coefficients measuring covariate effects at the point level are modified under aggregation. Changing the level of aggregation can lead to completely different conclusions about exposure–effect relationships, a phenomenon often referred to as ecological bias. With partial knowledge of the within-area distribution of the exposure variable, the notion of maximum entropy can be used to approximate that part of the distribution that is unknown. From the approximation, an expression for the ecological bias is obtained; simulations and an example show that the maximum-entropy approximation is often better than other commonly used approximations.