Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

Cluster analysis is the automated search for groups of related observations in a dataset. Most clustering done in practice is based largely on heuristic but intuitively reasonable procedures, and most clustering methods available in commercial software are also of this type. However, there is little systematic guidance associated with these methods for solving important practical questions that arise in cluster analysis, such as how many clusters are there, which clustering method should be used, and how should outliers be handled. We review a general methodology for model-based clustering that provides a principled statistical approach to these issues. We also show that this can be useful for other problems in multivariate analysis, such as discriminant analysis and multivariate density estimation. We give examples from medical diagnosis, minefield detection, cluster recovery from noisy data, and spatial density estimation. Finally, we mention limitations of the methodology and discuss recent development...

https://calvino.polito.it/~gasparini/teaching/fraley2002.pdf

Model-Based Clustering, Discriminant Analysis, and Density Estimation

We utilize Sloan Digital Sky Survey imaging and spectroscopy of ~53,000 star-forming galaxies at z ~ 0.1 to study the relation between stellar mass and gas-phase metallicity. We derive gas-phase oxygen abundances and stellar masses using new techniques that make use of the latest stellar evolutionary synthesis and photoionization models. We find a tight (?0.1 dex) correlation between stellar mass and metallicity spanning over 3 orders of magnitude in stellar mass and a factor of 10 in metallicity. The relation is relatively steep from 108.5 to 1010.5 M? h, in good accord with known trends between luminosity and metallicity, but flattens above 1010.5 M?. We use indirect estimates of the gas mass based on the H? luminosity to compare our data to predictions from simple closed box chemical evolution models. We show that metal loss is strongly anticorrelated with baryonic mass, with low-mass dwarf galaxies being 5 times more metal depleted than L* galaxies at z ~ 0.1. Evidence for metal depletion is not confined to dwarf galaxies but is found in galaxies with masses as high as 1010 M?. We interpret this as strong evidence of both the ubiquity of galactic winds and their effectiveness in removing metals from galaxy potential wells.

https://iopscience.iop.org/article/10.1086/423264/pdf

The Origin of the Mass-Metallicity Relation: Insights from 53,000 Star-forming Galaxies in the Sloan Digital Sky Survey

We consider the problem of determining the structure of clustered data, without prior knowledge of the number of clusters or any other information about their composition. Data are represented by a mixture model in which each component corresponds to a different cluster. Models with varying geometric properties are obtained through Gaussian components with different parametrizations and cross-cluster constraints. Noise and outliers can be modelled by adding a Poisson process component. Partitions are determined by the expectation-maximization (EM) algorithm for maximum likelihood, with initial values from agglomerative hierarchical clustering. Models are compared using an approximation to the Bayes factor based on the Bayesian information criterion (BIC); unlike significance tests, this allows comparison of more than two models at the same time, and removes the restriction that the models compared be nested. The problems of determining the number of clusters and the clustering method are solved simultaneously by choosing the best model. Moreover, the EM result provides a measure of uncertainty about the associated classification of each data point. Examples are given, showing that this approach can give performance that is much better than standard procedures, which often fail to identify groups that are either overlapping or of varying sizes and shapes.

https://www.stat.washington.edu/raftery/Research/PDF/fraley1998.pdf

How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis

Fitting a line to a bivariate dataset can be a deceptively complex problem, and there has been much debate on this issue in the literature. In this review, we describe for the practitioner the essential features of line-fitting methods for estimating the relationship between two variables: what methods are commonly used, which method should be used when, and how to make inferences from these lines to answer common research questions. A particularly important point for line-fitting in allometry is that usually, two sources of error are present (which we call measurement and equation error), and these have quite different implications for choice of line-fitting method. As a consequence, the approach in this review and the methods presented have subtle but important differences from previous reviews in the biology literature. Linear regression, major axis and standardised major axis are alternative methods that can be appropriate when there is no measurement error. When there is measurement error, this often needs to be estimated and used to adjust the variance terms in formulae for line-fitting. We also review line-fitting methods for phylogenetic analyses. Methods of inference are described for the line-fitting techniques discussed in this paper. The types of inference considered here are testing if the slope or elevation equals a given value, constructing confidence intervals for the slope or elevation, comparing several slopes or elevations, and testing for shift along the axis amongst several groups. In some cases several methods have been proposed in the literature. These are discussed and compared. In other cases there is little or no previous guidance available in the literature. Simulations were conducted to check whether the methods of inference proposed have the intended coverage probability or Type I error. We identified the methods of inference that perform well and recommend the techniques that should be adopted in future work.

/pdf/bivariate-line-fitting-methods-for-allometry-3qyicyo7mh.pdf

Bivariate line-fitting methods for allometry.

A wide variety of least-squares linear regression procedures used in observational astronomy, particularly investigations of the cosmic distance scale, are presented and discussed. The classes of linear models considered are (1) unweighted regression lines, with bootstrap and jackknife resampling; (2) regression solutions when measurement error, in one or both variables, dominates the scatter; (3) methods to apply a calibration line to new data; (4) truncated regression models, which apply to flux-limited data sets; and (5) censored regression models, which apply when nondetections are present. For the calibration problem we develop two new procedures: a formula for the intercept offset between two parallel data sets, which propagates slope errors from one regression to the other; and a generalization of the Working-Hotelling confidence bands to nonstandard least-squares lines. They can provide improved error analysis for Faber-Jackson, Tully-Fisher, and similar cosmic distance scale relations.

Linear regression in astronomy. II

A multivariate analysis of gamma-ray burst (GRB) bulk properties is presented to discriminate between distinct classes of GRBs. Several variables representing burst duration, Nuence, and spectral hardness are considered. Two multivariate clustering procedures are used on a sample of 797 bursts from the Third BATSE Catalog, a nonparametric average linkage hierarchical agglomerative clustering pro- cedure validated with WilksI "* and other multivariate analysis of variance tests and a parametric maximum likelihood modelEbased clustering procedure assuming multinormal populations calculated with the Expectation-Maximization algorithm and validated with the Bayesian Information Criterion. The two methods yield very similar results. The BATSE GRB population consists of three classes with the following duration/Nuence/spectrum bulk properties: class I with long/bright/soft bursts, class II with short/faint/hard bursts, and class III with intermediate/intermediate/soft bursts. One outlier due to spu- rious data is also present. Classes I and II correspond to those reported by Kouveliotou et al., but class III is clearly de-ned here for the -rst time. Subject headings: gamma rays: bursts E methods: data analysis E methods: statistical

/pdf/three-types-of-gamma-ray-bursts-2nrlcgnsz5.pdf

Three Types of Gamma-Ray Bursts

Despite centuries of close association, statistics and astronomy are surprisingly distant today. Most observational astronomical research relies on an inadequate toolbox of methodological tools. Yet the needs are substantial: astronomy encounters sophisticated problems involving sampling theory, survival analysis, multivariate classication and analysis, time series analysis, wavelet analysis, spatial point processes, nonlinear regression, bootstrap resampling and model selection. We review the recent resurgence of astrostatistical research, and outline new challenges raised by the emerging Virtual Observatory. Our essay ends with a list of research challenges and infrastructure for astrostatistics in the coming decade. 1. THE GLORIOUS HISTORY OF ASTRONOMY AND STATISTICS Astronomy is perhaps the oldest observational science 1 . The eort to understand the mysterious luminous objects in the sky has been an important element of human culture for at least 10 4 years. Quantitative measurements of celestial phenomena were carried out by many ancient civilizations. The classical Greeks were not active observers but were unusually creative in the applications of mathematical principles to astronomy. The geometric models of the Platonists with crystalline spheres spinning around the static Earth were elaborated in detail, and this model endured in Europe for 15 centuries. But it was another Greek natural philosopher, Hipparchus, who made one of the rst applications of mathematical principles that we now consider to be in the realm of statistics. Finding scatter in Bablylonian measurements of the length of a year, dened as the time between solstices, he took the middle of the range { rather than the mean or median { for the best value. This is but one of many discussions of statistical issues in the history of astronomy. Ptolemy estimated parameters of a non-linear cosmological model using a minimax goodness-of-t method. Al-Biruni discussed the dangers of propagating errors from inaccurate instruments and inattentive observers. While some Medieval scholars advised against the acquisition of repeated measurements, fearing that errors would compound rather than compensate for each other, the usefulnes of the mean to increase precision was demonstrated with great success by Tycho Brahe. During the 19th century, several elements of modern mathematical statistics were developed in the context

/pdf/statistical-challenges-in-modern-astronomy-3f5ao6tqtp.pdf

Statistical Challenges in Modern Astronomy

A multivariate analysis of gamma-ray burst (GRB) bulk properties is presented to discriminate between distinct classes of GRBs. Several variables representing burst duration, fluence and spectral hardness are considered. Two multivariate clustering procedures are used on a sample of 797 bursts from the Third BATSE Catalog: a nonparametric average linkage hierarchical agglomerative clustering procedure validated with Wilks' $\Lambda^*$ and other MANOVA tests; and a parametric maximum likelihood model-based clustering procedure assuming multinormal populations calculated with the EM Algorithm and validated with the Bayesian Information Criterion. 
The two methods yield very similar results. The BATSE GRB population consists of three classes with the following Duration/Fluence/Spectrum bulk properties: Class I with long/bright/intermediate bursts, Class II with short/hard/faint bursts, and Class III with intermediate/intermediate/soft bursts. One outlier with poor data is also present. Classes I and II correspond to those reported by Kouveliotou et al. (1993), but Class III is clearly defined here for the first time.

/pdf/three-types-of-gamma-ray-bursts-2zom2xcsm9.pdf

Three types of gamma-ray bursts

Efron (1979, 1982), in his treatment of the bootstrap, discusses its use for estimation of the asymptotic variance of the sample median, in the sampling situation of independent and identically distributed random variables with common distribution function $F$ having a positive derivative continuous in a neighborhood of the true median $\mu$. The natural conjecture that the bootstrap variance estimator converges almost surely to the asymptotic variance is shown by an example to be false unless a tail condition is imposed on $F$. We prove that such strong convergence does hold under the fairly nonrestrictive condition that $E\lbrack\mid X^\alpha\rbrack 0$.

/pdf/a-note-on-bootstrapping-the-sample-median-42v9u0rdxu.pdf

Gutti Jogesh Babu

Papers

Linear regression in astronomy. II

Three Types of Gamma-Ray Bursts

Statistical Challenges in Modern Astronomy

Three types of gamma-ray bursts

A Note on Bootstrapping the Sample Median