There is, I think, something ethereal about i —the square root of minus one. I remember first hearing about it at school. It seemed an odd beast at that time—an intruder hovering on the edge of reality.

Usually familiarity dulls this sense of the bizarre, but in the case of i it was the reverse: over the years the sense of its surreal nature intensified. It seemed that it was impossible to write mathematics that described the real world in …

I and i

General Strategies.- Regression Models.- Classification Models.- Other Considerations.- Appendix.- References.- Indices.

Applied Predictive Modeling

The author's preface gives an outline: "This book is about weakconvergence methods in metric spaces, with applications sufficient to show their power and utility. The Introduction motivates the definitions and indicates how the theory will yield solutions to problems arising outside it. Chapter 1 sets out the basic general theorems, which are then specialized in Chapter 2 to the space C[0, l ] of continuous functions on the unit interval and in Chapter 3 to the space D [0, 1 ] of functions with discontinuities of the first kind. The results of the first three chapters are used in Chapter 4 to derive a variety of limit theorems for dependent sequences of random variables. " The book develops and expands on Donsker's 1951 and 1952 papers on the invariance principle and empirical distributions. The basic random variables remain real-valued although, of course, measures on C[0, l ] and D[0, l ] are vitally used. Within this framework, there are various possibilities for a different and apparently better treatment of the material. More of the general theory of weak convergence of probabilities on separable metric spaces would be useful. Metrizability of the convergence is not brought up until late in the Appendix. The close relation of the Prokhorov metric and a metric for convergence in probability is (hence) not mentioned (see V. Strassen, Ann. Math. Statist. 36 (1965), 423-439; the reviewer, ibid. 39 (1968), 1563-1572). This relation would illuminate and organize such results as Theorems 4.1, 4.2 and 4.4 which give isolated, ad hoc connections between weak convergence of measures and nearness in probability. In the middle of p. 16, it should be noted that C*(S) consists of signed measures which need only be finitely additive if 5 is not compact. On p. 239, where the author twice speaks of separable subsets having nonmeasurable cardinal, he means "discrete" rather than "separable." Theorem 1.4 is Ulam's theorem that a Borel probability on a complete separable metric space is tight. Theorem 1 of Appendix 3 weakens completeness to topological completeness. After mentioning that probabilities on the rationals are tight, the author says it is an

Convergence of probability measures

Used to estimate the risk of an estimator or to perform model selection, cross-validation is a widespread strategy because of its simplicity and its apparent universality. Many results exist on the model selection performances of cross-validation procedures. This survey intends to relate these results to the most recent advances of model selection theory, with a particular emphasis on distinguishing empirical statements from rigorous theoretical results. As a conclusion, guidelines are provided for choosing the best cross-validation procedure according to the particular features of the problem in hand.

/pdf/a-survey-of-cross-validation-procedures-for-model-selection-2haa63bftz.pdf

A survey of cross-validation procedures for model selection

In undergoing this life, many people always try to do and get the best. New knowledge, experience, lesson, and everything that can improve the life will be done. However, many people sometimes feel confused to get those things. Feeling the limited of experience and sources to be better is one of the lacks to own. However, there is a very simple thing that can be done. This is what your teacher always manoeuvres you to do this one. Yeah, reading is the answer. Reading a book as this a distribution free theory of nonparametric regression springer series in statistics and other references can enrich your life quality. How can it be?

A Distribution-Free Theory of Nonparametric Regression (Springer Series in Statistics)

Assuming that a smoothness condition and a suitable restriction on the structure of the regression function hold, it is shown that least squares estimates based on multilayer feedforward neural networks are able to circumvent the curse of dimensionality in nonparametric regression. The proof is based on new approximation results concerning multilayer feedforward neural networks with bounded weights and a bounded number of hidden neurons. The estimates are compared with various other approaches by using simulated data.

/pdf/on-deep-learning-as-a-remedy-for-the-curse-of-dimensionality-fhqxbdacal.pdf

On deep learning as a remedy for the curse of dimensionality in nonparametric regression

The determination of the fetal D status from maternal plasma for decision making on Rh prophylaxis is feasible.

Probability estimation for binary and multicategory outcome using logistic and multinomial logistic regression has a long-standing tradition in biostatistics. However, biases may occur if the model is misspecified. In contrast, outcome probabilities for individuals can be estimated consistently with machine learning approaches, including k-nearest neighbors (k-NN), bagged nearest neighbors (b-NN), random forests (RF), and support vector machines (SVM). Because machine learning methods are rarely used by applied biostatisticians, the primary goal of this paper is to explain the concept of probability estimation with these methods and to summarize recent theoretical findings. Probability estimation in k-NN, b-NN, and RF can be embedded into the class of nonparametric regression learning machines; therefore, we start with the construction of nonparametric regression estimates and review results on consistency and rates of convergence. In SVMs, outcome probabilities for individuals are estimated consistently by repeatedly solving classification problems. For SVMs we review classification problem and then dichotomous probability estimation. Next we extend the algorithms for estimating probabilities using k-NN, b-NN, and RF to multicategory outcomes and discuss approaches for the multicategory probability estimation problem using SVM. In simulation studies for dichotomous and multicategory dependent variables we demonstrate the general validity of the machine learning methods and compare it with logistic regression. However, each method fails in at least one simulation scenario. We conclude with a discussion of the failures and give recommendations for selecting and tuning the methods. Applications to real data and example code are provided in a companion article (doi:10.1002/bimj.201300077).

Probability estimation with machine learning methods for dichotomous and multicategory outcome: Theory

In this paper, we introduce the so-called hierarchical interaction models, where we assume that the computation of the value of a function $m: { \mathbb {R}^{d}}\rightarrow \mathbb {R}$ is done in several layers, where in each layer a function of at most $d^{*}$ inputs computed by the previous layer is evaluated. We investigate two different regression estimates based on polynomial splines and on neural networks, and show that if the regression function satisfies a hierarchical interaction model and all occurring functions in the model are smooth, the rate of convergence of these estimates depends on $d^{*}$ (and not on $d$ ). Hence, in this case, the estimates can achieve good rate of convergence even for large $d$ , and are in this sense able to circumvent the so-called curse of dimensionality.

Michael Kohler

Papers

A Distribution-Free Theory of Nonparametric Regression (Springer Series in Statistics)

On deep learning as a remedy for the curse of dimensionality in nonparametric regression

The determination of the fetal D status from maternal plasma for decision making on Rh prophylaxis is feasible.

Probability estimation with machine learning methods for dichotomous and multicategory outcome: Theory

Nonparametric Regression Based on Hierarchical Interaction Models