Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.

The Nature of Statistical Learning Theory

We propose a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams [6], and Hof-mann's aspect model, also known as probabilistic latent semantic indexing (pLSI) [3]. In the context of text modeling, our model posits that each document is generated as a mixture of topics, where the continuous-valued mixture proportions are distributed as a latent Dirichlet random variable. Inference and learning are carried out efficiently via variational algorithms. We present empirical results on applications of this model to problems in text modeling, collaborative filtering, and text classification.

/pdf/latent-dirichlet-allocation-rkwbqihm3p.pdf

Latent Dirichlet Allocation

THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

The application of maximum likelihood techniques to the estimation of evolutionary trees from nucleic acid sequence data is discussed. A computationally feasible method for finding such maximum likelihood estimates is developed, and a computer program is available. This method has advantages over the traditional parsimony algorithms, which can give misleading results if rates of evolution differ in different lineages. It also allows the testing of hypotheses about the constancy of evolutionary rates by likelihood ratio tests, and gives rough indication of the error of the estimate of the tree.

Evolutionary trees from DNA sequences: A maximum likelihood approach

An essential textbook for any student or researcher in biology needing to design experiments, sample programs or analyse the resulting data The text begins with a revision of estimation and hypothesis testing methods, covering both classical and Bayesian philosophies, before advancing to the analysis of linear and generalized linear models Topics covered include linear and logistic regression, simple and complex ANOVA models (for factorial, nested, block, split-plot and repeated measures and covariance designs), and log-linear models Multivariate techniques, including classification and ordination, are then introduced Special emphasis is placed on checking assumptions, exploratory data analysis and presentation of results The main analyses are illustrated with many examples from published papers and there is an extensive reference list to both the statistical and biological literature The book is supported by a website that provides all data sets, questions for each chapter and links to software

/pdf/experimental-design-and-data-analysis-for-biologists-8ybva8d53x.pdf

Experimental Design and Data Analysis for Biologists

Several problems involving multivariate generalized Bayes estimators are investigated. First, a characterization of admissible estimators as generalized Bayes estimators is developed for certain multivariate exponential families and quadratic loss. The problem of verifying whether or not an estimator is generalized Bayes is also considered. Next, an important class of estimators for a multivariate normal mean is considered. (The class includes many minimax, empirical Bayes, and ridge regression estimators of current interest.) Necessary conditions are developed for an estimator in this class to be "nearly" generalized Bayes, in the sense that if it were properly smoothed, it would be generalized Bayes. An application to adaptive ridge regression is given. The paper concludes with the development of an asymptotic approximation to generalized Bayes estimators for general losses and location vector densities. Using this approximation, weakened versions of the above results are obtained for general losses and densities.

/pdf/generalized-bayes-estimators-in-multivariate-problems-2n379nilmn.pdf

Generalized Bayes Estimators in Multivariate Problems

The CRASH computer model simulates the effect of a vehicle colliding against different barrier types. If it accurately represents real vehicle crashworthiness, the computer model can be of great value in various aspects of vehicle design, such as the setting of timing of air bag releases. The goal of this study is to address the problem of validating the computer model for such design goals, based on utilizing computer model runs and experimental data from real crashes. This task is complicated by the fact that (i) the output of this model consists of smooth functional data, and (ii) certain types of collision have very limited data. We address problem (i) by extending existing Gaussian process-based methodology developed for models that produce real-valued output, and resort to Bayesian hierarchical modeling to attack problem (ii). Additionally, we show how to formally test if the computer model reproduces reality. Supplemental materials for the article are available online.

Predicting Vehicle Crashworthiness: Validation of Computer Models for Functional and Hierarchical Data

Item response theory (IRT) models have been widely used in educational measurement testing When there are repeated observations available for individuals through time, a dynamic structure for the latent trait of ability needs to be incorporated into the model, to accommodate changes in ability Other complications that often arise in such settings include a violation of the common assumption that test results are conditionally independent, given ability and item difficulty, and that test item difficulties may be partially specified, but subject to uncertainty Focusing on time series dichotomous response data, a new class of state space models, called Dynamic Item Response (DIR) models, is proposed The models can be applied either retrospectively to the full data or on-line, in cases where real-time prediction is needed The models are studied through simulated examples and applied to a large collection of reading test data obtained from MetaMetrics, Inc

Bayesian analysis of dynamic item response models in educational testing

Background Several medical articles discuss methods of constructing confidence intervals for single proportions and the likelihood ratio, but scant attention has been given to the systematic study

Intervals for posttest probabilities: a comparison of 5 methods.

Objective Bayesian inference for the multivariate normal distribution is illustrated, using different types of formal objective priors (Jeffreys, invariant, reference and matching), different modes of inference (Bayesian and frequentist), and different criteria involved in selecting optimal objective priors (ease of computation, frequentist performance, marginalization paradoxes, and decision-theoretic evaluation).

/pdf/objective-bayesian-analysis-for-the-multivariate-normal-exn2b6o5k1.pdf

James O. Berger

Papers

Generalized Bayes Estimators in Multivariate Problems

Predicting Vehicle Crashworthiness: Validation of Computer Models for Functional and Hierarchical Data

Bayesian analysis of dynamic item response models in educational testing

Intervals for posttest probabilities: a comparison of 5 methods.

Objective Bayesian Analysis for the Multivariate Normal Model