There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

Publishing data about individuals without revealing sensitive information about them is an important problem. In recent years, a new definition of privacy called k-anonymity has gained popularity. In a k-anonymized dataset, each record is indistinguishable from at least k − 1 other records with respect to certain identifying attributes.In this article, we show using two simple attacks that a k-anonymized dataset has some subtle but severe privacy problems. First, an attacker can discover the values of sensitive attributes when there is little diversity in those sensitive attributes. This is a known problem. Second, attackers often have background knowledge, and we show that k-anonymity does not guarantee privacy against attackers using background knowledge. We give a detailed analysis of these two attacks, and we propose a novel and powerful privacy criterion called e-diversity that can defend against such attacks. In addition to building a formal foundation for e-diversity, we show in an experimental evaluation that e-diversity is practical and can be implemented efficiently.

/pdf/l-diversity-privacy-beyond-k-anonymity-54qdj1526d.pdf

L-diversity: Privacy beyond k-anonymity

Unit root tests for panel data

The Analysis of Variance.

8. Subset Selection in Regression (Monographs on Statistics and Applied Probability, no. 40). By A. J. Miller. ISBN 0 412 35380 6. Chapman and Hall, London, 1990. 240 pp. £25.00.

Subset Selection in Regression

“Working the Web,” “Reporting a Disease Outbreak,” and “What Makes a Good Reporter?” Given the extensive collection of information, how could anyone fail to learn something useful from reading this book? There are a few statistical misinterpretations. The most egregious is a misunderstanding of the meaning of p values, presenting the conditional probability in the wrong direction: “All that p values represent is the probability that the experimental results could be the product of chance alone. In 20 experiments that report a positive  nding at a p value of .05, one of these  ndings will be the result of chance alone” (p. 21). Unfortunately, the interpretation is not so simple. For those who think that that is a correct interpretation, consider 200 tests of hypotheses in which 100 have true null hypotheses and 100 have true alternative hypotheses. Furthermore, suppose that the power for the 100 situations where the alternative hypothesis is true is relatively low at .15, and that the null hypothesis is rejected in 15 of the 100 cases. Suppose also that exactly .05 or 5 of the true null hypotheses are rejected. Then there are 20 experiments reporting positive  ndings, but 5 of them (25% ), not 1 of them (5% ), are the result of chance alone. In general, the percent of false positives in a collection of studies depends on how many of the studies were investigating a real effect, and how likely each of those studies were to  nd that effect (called the power of the study). At the extreme, if all studies in the collection were investigating a real effect, there could be no false positives and all signi cant  ndings would be true positives. The most interesting stories in science involve controversial  ndings. As one who sometimes works in a controversial research area rife with bad reporting (parapsychology), I was particularly heartened by two tips in a list of 18 that the authors provide to reporters in chapter 9 (both on page 144). The  rst, “don’t be afraid to challenge an expert,” cautions that journalists are too reluctant to challenge “those who use the authority of science to shape the public view.” The second, “broaden your concept of good sources,” notes that “sometimes the people who really know what’s going on aren’t the conventional experts.” Too often, reporters assume that a scientist who is an expert in one area is quali ed to comment on an area in which he or she at best has read none of the original research, and at worst won’t read any of the original research because of a preconceived bias. In summary, this book is interesting for statisticians because it offers a look at statistical studies from the perspective of science journalists who need to condense and explain results to the public. It is interesting for science writers because it introduces them to a wide range of statistical issues that they need to understand for in-depth reporting. Reading this book in conjunction with one of the expanding number of texts on statistical thinking would be very useful preparation for investigative science reporters. Finally, the book is interesting for the general public because it informs on a wide range of interesting statistical, medical, social and environmental subjects.

An Introduction to Probability Theory and Mathematical Statistics

Let $y$ be a single observation on a $p \times 1$ random vector which is distributed according to the multivariate normal distribution with mean vector $\theta$ and covariance matrix $I$. Consider the problem of estimating $\theta$ when the loss function is the sum of the squared errors in estimating the individual components of $\theta$. Let $G$ be a $p \times p$ real matrix. Then we will prove that the estimate $Gy$ is admissible if and only if $G$ is symmetric, and the characteristic roots of $G, g_i$ say, $i = 1, 2, \cdots, p$, satisfy $0 \leqq g_i \leqq 1$, with equality at one for at most two of the roots. The proof concerning the characteristic roots uses the results of Karlin [4] and James and Stein [3]. We give two proofs of the fact that $Gy$ is inadmissible when $G$ is asymmetrical. In the first proof we give an estimate $G^\ast y$ that is better than $Gy$, where $G^\ast$ is a symmetric matrix. This not only adds to the practicality of the result, but also enables us to resolve the question of which estimates are admissible in the restricted class of estimates of the form $Gy$. The method of the second proof, which utilizes a theorem of Sacks [6], leads to the following finding: If $Gy$ is admissible, then $Gy$ must be a generalized Bayes procedure, where the unique generalized prior distribution must be either a multivariate normal distribution with mean vector zero and a specified covariance matrix determined by $G$, or the product of a distribution which is multivariate normal over a subspace of the parameter space and a distribution which is uniformly distributed over a subspace of the parameter space. This latter finding generalizes the well known one dimensional case. In Section 2 then the main results are proved. In Section 3 some remarks concerned with generalizing the main results are given. We remark here that the decision theory terminology used is more or less that of Blackwell and Girshick [1].

/pdf/all-admissible-linear-estimates-of-the-mean-vector-3xgwx64dao.pdf

All Admissible Linear Estimates of the Mean Vector

Consider the problem of estimating a common mean of two independent normal distributions, each with unknown variances. Note that the problem of recovery of interblock information in balanced incomplete blocks designs is such a problem. Suppose a random sample of size $m$ is drawn from the first population and a random sample of size $n$ is drawn from the second population. We first show that the sample mean of the first population can be improved on (with an unbiased estimator having smaller variance), provided $m \geqq 2$ and $n \geqq 3$. The method of proof is applicable to the recovery of information problem. For that problem, it is shown that interblock information could be used provided $b \geqq 4$. Furthermore for the case $b = t = 3$, or in the common mean problem, where $n = 2$, it is shown that the prescribed estimator does not offer improvement. Some of the results for the common mean problem are extended to the case of $K$ means. Results similar to some of those obtained for point estimation, are also obtained for confidence estimation.

/pdf/point-and-confidence-estimation-of-a-common-mean-and-2y6fgtlpml.pdf

Point and Confidence Estimation of a Common Mean and Recovery of Interblock Information

Two stage conditionally unbiased estimators of the selected mean

Let Xi1, Xi2, …, Xin, i = 1, 2, be a pair of random samples from populations which are normally distributed with means θi, and common known variance τ2. The problem is to estimate the function φ(θ1, θ2) = maximum (θ1, θ2). In this paper we consider five different estimators (or sets of estimators) for φ(θ1, θ2) and evaluate their biases and mean square errors. The estimators are (i) φ(X1, X2), where Xi is the sample mean of the ith sample; (ii) the analogue of the Pitman estimator, i.e. the a posteriori expected value of φ(θ1, θ2) when the generalized prior distribution is the uniform distribution on two dimensional space; (iii) a class of estimators which are generalized Bayes with respect to generalized priors which are products of uniform and normal priors; (iv) hybrid estimators, i.e. those which estimate by (X1 + X2)/2 when |X1 – X2| is small, and estimate by φ(X1, X2) when |X1 − X2| is large; (v) maximum likelihood estimator. The bias and mean square errors for these estimators are tabled, graphed, ...

Arthur Cohen

Papers

An Introduction to Probability Theory and Mathematical Statistics

All Admissible Linear Estimates of the Mean Vector

Point and Confidence Estimation of a Common Mean and Recovery of Interblock Information

Two stage conditionally unbiased estimators of the selected mean

Estimation of the larger of two normal means