THE DESIGN AND ANALYSIS OF EXPERIMENTS. By Oscar Kempthorne. New York, John Wiley and Sons, Inc., 1952. 631 pp. $8.50. This book by a teacher of statistics (as well as a consultant for \"experimenters\") is a comprehensive study of the philosophical background for the statistical design of experiment. It is necessary to have some facility with algebraic notation and manipulation to be able to use the volume intelligently. The problems are presented from the theoretical point of view, without such practical examples as would be helpful for those not acquainted with mathematics. The mathematical justification for the techniques is given. As a somewhat advanced treatment of the design and analysis of experiments, this volume will be interesting and helpful for many who approach statistics theoretically as well as practically. With emphasis on the \"why,\" and with description given broadly, the author relates the subject matter to the general theory of statistics and to the general problem of experimental inference. MARGARET J. ROBERTSON

The Design and Analysis of Experiments

The goal of machine learning is to program computers to use example data or past experience to solve a given problem. Many successful applications of machine learning exist already, including systems that analyze past sales data to predict customer behavior, optimize robot behavior so that a task can be completed using minimum resources, and extract knowledge from bioinformatics data. Introduction to Machine Learning is a comprehensive textbook on the subject, covering a broad array of topics not usually included in introductory machine learning texts. In order to present a unified treatment of machine learning problems and solutions, it discusses many methods from different fields, including statistics, pattern recognition, neural networks, artificial intelligence, signal processing, control, and data mining. All learning algorithms are explained so that the student can easily move from the equations in the book to a computer program. The text covers such topics as supervised learning, Bayesian decision theory, parametric methods, multivariate methods, multilayer perceptrons, local models, hidden Markov models, assessing and comparing classification algorithms, and reinforcement learning. New to the second edition are chapters on kernel machines, graphical models, and Bayesian estimation; expanded coverage of statistical tests in a chapter on design and analysis of machine learning experiments; case studies available on the Web (with downloadable results for instructors); and many additional exercises. All chapters have been revised and updated. Introduction to Machine Learning can be used by advanced undergraduates and graduate students who have completed courses in computer programming, probability, calculus, and linear algebra. It will also be of interest to engineers in the field who are concerned with the application of machine learning methods. Adaptive Computation and Machine Learning series

Introduction to Machine Learning

Information Theory and Reliable Communication

5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

: This thesis applies neural network feature selection techniques to multivariate time series data to improve prediction of a target time series. Two approaches to feature selection are used. First, a subset enumeration method is used to determine which financial indicators are most useful for aiding in prediction of the S&P 500 futures daily price. The candidate indicators evaluated include RSI, Stochastics and several moving averages. Results indicate that the Stochastics and RSI indicators result in better prediction results than the moving averages. The second approach to feature selection is calculation of individual saliency metrics. A new decision boundary-based individual saliency metric, and a classifier independent saliency metric are developed and tested. Ruck's saliency metric, the decision boundary based saliency metric, and the classifier independent saliency metric are compared for a data set consisting of the RSI and Stochastics indicators as well as delayed closing price values. The decision based metric and the Ruck metric results are similar, but the classifier independent metric agrees with neither of the other metrics. The nine most salient features, determined by the decision boundary based metric, are used to train a neural network and the results are presented and compared to other published results. (AN)

/pdf/nonlinear-time-series-analysis-1j3a35n4y3.pdf

Nonlinear Time Series Analysis.

Stopping sets determine the performance of low-density parity-check (LDPC) codes under iterative decoding over erasure channels. We derive several results on the asymptotic behavior of stopping sets in Tanner-graph ensembles, including the following. An expression for the normalized average stopping set distribution, yielding, in particular, a critical fraction of the block length above which codes have exponentially many stopping sets of that size. A relation between the degree distribution and the likely size of the smallest nonempty stopping set, showing that for a /spl radic/1-/spl lambda/'(0)/spl rho/'(1) fraction of codes with /spl lambda/'(0)/spl rho/'(1) 2, the smallest nonempty stopping set is linear in the block length. Bounds on the average block error probability as a function of the erasure probability /spl epsi/, showing in particular that for codes with lowest variable degree 2, if /spl epsi/ is below a certain threshold, the asymptotic average block error probability is 1-/spl radic/1-/spl lambda/'(0)/spl rho/'(1)/spl epsi/.

http://fleece.ucsd.edu/~alon/papers/stp_set_dst.pdf

Stopping set distribution of LDPC code ensembles

Online anomaly detection is an important step in data center management, requiring light-weight techniques that provide sufficient accuracy for subsequent diagnosis and management actions. This paper presents statistical techniques based on the Tukey and Relative Entropy statistics, and applies them to data collected from a production environment and to data captured from a testbed for multi-tier web applications running on server class machines. The proposed techniques are lightweight and improve over standard Gaussian assumptions in terms of performance.

/pdf/statistical-techniques-for-online-anomaly-detection-in-data-2ganyq94ad.pdf

Statistical techniques for online anomaly detection in data centers

Recent work has related the error probability of iterative decoding over erasure channels to the presence of stopping sets in the Tanner graph of the code used. In particular, it was shown that the smallest number of uncorrected erasures is the size of the graph's smallest stopping set. Relating stopping sets and girths, we consider the size /spl sigma/(d,g) of the smallest stopping set in any bipartite graph of girth g and left degree d. For g/spl les/8 and any d, we determine /spl sigma/(d,g) exactly. For larger gs we bound /spl sigma/(d,g) in terms of d, showing that for fixed d, /spl sigma/(d,g) grows exponentially with g. Since constructions of high-girth graphs are known, one can therefore design codes with good erasure-correction guarantees under iterative decoding.

Stopping sets and the girth of Tanner graphs

We consider the problem of estimating the distribution underlying an observed sample of data. Instead of maximum likelihood, which maximizes the probability of the observed values, we propose a different estimate, the high-profile distribution, which maximizes the probability of the observed profile---the number of symbols appearing any given number of times. We determine the high-profile distribution of several data samples, establish some of its general properties, and show that when the number of distinct symbols observed is small compared to the data size, the high-profile and maximum-likelihood distributions are roughly the same, but when the number of symbols is large, the distributions differ, and high-profile better explains the data.

On modeling profiles instead of values

Consider a binary string x of length n transmitted m times over a memoryless channel that randomly inserts, deletes and flips bits. We consider the problem of reconstructing x from the collection of m binary strings received at the output of the channel. We present an algorithm for this problem and show that, as n tends to infinity, for almost all transmitted strings, the algorithm is guaranteed to reconstruct the transmitted string with probability 1 - o(1), if the channel's flip probability p qi,qd = O(1/log n), and m = θ(log n). Our algorithm improves on [1] which applies to channels that only delete, and on [2] where qi,qd = O(1/(log n)2). We also show that to reconstruct most strings reliably over any channel with a constant flip probability m = Ω(log n) transmissions are necessary, and therefore our algorithm is efficient in this sense.

Krishnamurthy Viswanathan

Papers

Stopping set distribution of LDPC code ensembles

Statistical techniques for online anomaly detection in data centers

Stopping sets and the girth of Tanner graphs

On modeling profiles instead of values

Improved string reconstruction over insertion-deletion channels