Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.

The Nature of Statistical Learning Theory

We propose a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams [6], and Hof-mann's aspect model, also known as probabilistic latent semantic indexing (pLSI) [3]. In the context of text modeling, our model posits that each document is generated as a mixture of topics, where the continuous-valued mixture proportions are distributed as a latent Dirichlet random variable. Inference and learning are carried out efficiently via variational algorithms. We present empirical results on applications of this model to problems in text modeling, collaborative filtering, and text classification.

/pdf/latent-dirichlet-allocation-rkwbqihm3p.pdf

Latent Dirichlet Allocation

THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

The application of maximum likelihood techniques to the estimation of evolutionary trees from nucleic acid sequence data is discussed. A computationally feasible method for finding such maximum likelihood estimates is developed, and a computer program is available. This method has advantages over the traditional parsimony algorithms, which can give misleading results if rates of evolution differ in different lineages. It also allows the testing of hypotheses about the constancy of evolutionary rates by likelihood ratio tests, and gives rough indication of the error of the estimate of the tree.

Evolutionary trees from DNA sequences: A maximum likelihood approach

An essential textbook for any student or researcher in biology needing to design experiments, sample programs or analyse the resulting data The text begins with a revision of estimation and hypothesis testing methods, covering both classical and Bayesian philosophies, before advancing to the analysis of linear and generalized linear models Topics covered include linear and logistic regression, simple and complex ANOVA models (for factorial, nested, block, split-plot and repeated measures and covariance designs), and log-linear models Multivariate techniques, including classification and ordination, are then introduced Special emphasis is placed on checking assumptions, exploratory data analysis and presentation of results The main analyses are illustrated with many examples from published papers and there is an extensive reference list to both the statistical and biological literature The book is supported by a website that provides all data sets, questions for each chapter and links to software

/pdf/experimental-design-and-data-analysis-for-biologists-8ybva8d53x.pdf

Experimental Design and Data Analysis for Biologists

This paper describes an efficient and systematic process for using geophysical computer model simulations to guide efforts in probabilistic hazard mapping. The framework being proposed requires the simultaneous construction of many ($10^2$--$10^4$) statistical emulators, e.g., cheap model surrogates. This paper describes an automation process for choosing designs for and fitting these emulators. Throughout the description of this process, several useful modifications to standard emulators are explored. Additionally, this approach enables a fast and flexible uncertainty quantification for multiple sources of aleatory variability (natural randomness) and epistemic uncertainty (uncertainty in geophysical and statistical models) in the context of probabilistic hazard mapping. This process is illustrated through an application to granular volcanic flows.

https://epubs.siam.org/doi/pdf/10.1137/120899285

Automating Emulator Construction for Geophysical Hazard Maps

Hierarchical modeling is wonderful and here to stay, but hyperparameter priors are often chosen in a casual fashion. Unfortunately, as the number of hyperparameters grows, the effects of casual choices can multiply, leading to considerably inferior performance. As an extreme, but not uncommon, example use of the wrong hyperparameter priors can even lead to impropriety of the posterior. For exchangeable hierarchical multivariate normal models, we first determine when a standard class of hierarchical priors results in proper or improper posteriors. We next determine which elements of this class lead to admissible estimators of the mean under quadratic loss; such considerations provide one useful guideline for choice among hierarchical priors. Finally, computational issues with the resulting posterior distributions are addressed.

/pdf/posterior-propriety-and-admissibility-of-hyperpriors-in-2u3il3tslu.pdf

Posterior propriety and admissibility of hyperpriors in normal hierarchical models

We develop and describe a Bayesian statistical analysis to solve the surface brightness equations for Cepheid distances and stellar properties. Our analysis provides a mathematically rigorous and objective solution to the problem, including immunity from Lutz-Kelker bias. We discuss the choice of priors, show the construction of the likelihood distribution, and give sampling algorithms in a Markov chain Monte Carlo approach for efficiently and completely sampling the posterior probability distribution. Our analysis averages over the probabilities associated with several models rather than attempting to pick the best model from several possible models. Using a sample of 13 Cepheids we demonstrate the method. We discuss diagnostics of the analysis and the effects of the astrophysical choices going into the model. We show that we can objectively model the order of Fourier polynomial fits to the light and velocity data. By comparison with theoretical models of Bono et al. we find that EU Tau and SZ Tau are overtone pulsators, most likely without convective overshoot. The period-radius and period-luminosity relations we obtain are shown to be compatible with those in the recent literature. Specifically, we find log(R) = (0.693 ± 0.037) [log(P) − 1.2] (2.042 ± 0.047) and Mv = −(2.690 ± 0.169) [log(P) − 1.2] − (4.699 ± 0.216).

https://www2.stat.duke.edu/~berger/papers/apjcepheid.pdf

A Bayesian Analysis of the Cepheid Distance Scale

In simultaneous estimation of normal means, it is shown that through use of the Stein effect surprisingly large gains of a Bayesian nature can be achieved, at little or no cost, if the prior information is misspecified. This provides a justification, in terms of robustness with respect to mis-specification of the prior, for employing the Stein effect, even when combining a priori independent problems (i.e., problems in which no empirical Bayes effects are obtainable). To study this issue, a class of minimax estimators that closely mimic the conjugate prior Bayes estimators is introduced.

James O. Berger

Papers

Automating Emulator Construction for Geophysical Hazard Maps

Posterior propriety and admissibility of hyperpriors in normal hierarchical models

A Bayesian Analysis of the Cepheid Distance Scale

Bayesian Robustness and the Stein Effect

Minimax estimation of a multivariate normal mean under arbitrary quadratic loss