Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.

The Nature of Statistical Learning Theory

We propose a generative model for text and other collections of discrete data that generalizes or improves on several previous models including naive Bayes/unigram, mixture of unigrams [6], and Hof-mann's aspect model, also known as probabilistic latent semantic indexing (pLSI) [3]. In the context of text modeling, our model posits that each document is generated as a mixture of topics, where the continuous-valued mixture proportions are distributed as a latent Dirichlet random variable. Inference and learning are carried out efficiently via variational algorithms. We present empirical results on applications of this model to problems in text modeling, collaborative filtering, and text classification.

/pdf/latent-dirichlet-allocation-rkwbqihm3p.pdf

Latent Dirichlet Allocation

THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

The application of maximum likelihood techniques to the estimation of evolutionary trees from nucleic acid sequence data is discussed. A computationally feasible method for finding such maximum likelihood estimates is developed, and a computer program is available. This method has advantages over the traditional parsimony algorithms, which can give misleading results if rates of evolution differ in different lineages. It also allows the testing of hypotheses about the constancy of evolutionary rates by likelihood ratio tests, and gives rough indication of the error of the estimate of the tree.

Evolutionary trees from DNA sequences: A maximum likelihood approach

An essential textbook for any student or researcher in biology needing to design experiments, sample programs or analyse the resulting data The text begins with a revision of estimation and hypothesis testing methods, covering both classical and Bayesian philosophies, before advancing to the analysis of linear and generalized linear models Topics covered include linear and logistic regression, simple and complex ANOVA models (for factorial, nested, block, split-plot and repeated measures and covariance designs), and log-linear models Multivariate techniques, including classification and ordination, are then introduced Special emphasis is placed on checking assumptions, exploratory data analysis and presentation of results The main analyses are illustrated with many examples from published papers and there is an extensive reference list to both the statistical and biological literature The book is supported by a website that provides all data sets, questions for each chapter and links to software

/pdf/experimental-design-and-data-analysis-for-biologists-8ybva8d53x.pdf

Experimental Design and Data Analysis for Biologists

Bayesian analysis incorporates different sources of information into a single analysis through Bayes theorem. When one or more of the sources of information are suspect (e.g., if the model assumed for the information is viewed as quite possibly being significantly flawed), there can be a concern that Bayes theorem allows this suspect information to overly influence the other sources of information. We consider a variety of situations in which this arises, and give methodological suggestions for dealing with the problem. After consideration of some pedagogical examples of the phenomenon, we focus on the interface of statistics and the development of complex computer models of processes. Three testbed computer models are considered, in which this type of issue arises.

/pdf/modularization-in-bayesian-analysis-with-emphasis-on-1a6uh7solw.pdf

Modularization in Bayesian analysis, with emphasis on analysis of computer models

In this paper, we show that the conditional frequentist method of testing a precise hypothesis can be made virtually equiva- lent to Bayesian testing. The conditioning strategy proposed by Berger, Brown and Wolpert in 1994, for the simple versus simple case, is gener- alized to testing a precise null hypothesis versus a composite alternative hypothesis. Using this strategy, both the conditional frequentist and the Bayesian will report the same error probabilities upon rejecting or ac- cepting. This is of considerable interest because it is often perceived that Bayesian and frequentist testing are incompatible in this situation. That they are compatible, when conditional frequentist testing is allowed, is a strong indication that the \wrong" frequentist tests are currently be- ing used for postexperimental assessment of accuracy. The new unied testing procedure is discussed and illustrated in several common testing situations.

/pdf/unified-frequentist-and-bayesian-testing-of-a-precise-aibi5e402d.pdf

Unified frequentist and Bayesian testing of a precise hypothesis

An Adaptive Importance Sampling (AIS) scheme is introduced to compute integrals of the form as a mechanical, yet flexible, way of dealing with the selection of parameters of the importance function. AIS starts with a rough estimate for the parameters λ of the importance function g , and runs importance sampling in an iterative way to continually update λ using only linear accumulation. Consistency of AIS is established. The efficiency of the algorithm is studied in three examples and found to be substantially superior to ordinary importance sampling.

Adaptive importance sampling in monte carlo integration

SUMMARY We consider the problem of comparing parametric models using a Bayesian approach. A new method of developing prior distributions for the model parameters is presented, called the expected-posterior prior approach. The idea is to define the priors for all models from a common underlying predictive distribution, in such a way that the resulting priors are amenable to modern Markov chain Monte Carlo computational techniques. The approach has subjective Bayesian and default Bayesian implementations, and overcomes the most significant impediment to Bayesian model selection, that of ensuring that prior distributions for the various models are appropriately compatible.

Expected‐posterior prior distributions for model selection

In this article Bayesian analysis for a Poly-Weibull distribution using informative priors is discussed. This distribution typically arises when the data is the minimum of several Weibull failure times from competing risks. To perform the Bayesian computations, simulation using the Gibbs sampler is suggested. This can be used to find posterior moments, the marginal posterior probability density function, and the predictive risk or reliability.

James O. Berger

Papers

Modularization in Bayesian analysis, with emphasis on analysis of computer models

Unified frequentist and Bayesian testing of a precise hypothesis

Adaptive importance sampling in monte carlo integration

Expected‐posterior prior distributions for model selection

Bayesian Analysis for the Poly-Weibull Distribution