Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

https://www.sfs.uni-tuebingen.de/~hbaayen/publications/baayenDavidsonBates.pdf

Mixed-effects modeling with crossed random effects for subjects and items

Collinearity refers to the non independence of predictor variables, usually in a regression-type analysis. It is a common feature of any descriptive ecological data set and can be a problem for parameter estimation because it inflates the variance of regression parameters and hence potentially leads to the wrong identification of relevant predictors in a statistical model. Collinearity is a severe problem when a model is trained on data from one region or time, and predicted to another with a different or unknown structure of collinearity. To demonstrate the reach of the problem of collinearity in ecology, we show how relationships among predictors differ between biomes, change over spatial scales and through time. Across disciplines, different approaches to addressing collinearity problems have been developed, ranging from clustering of predictors, threshold-based pre-selection, through latent variable methods, to shrinkage and regularisation. Using simulated data with five predictor-response relationships of increasing complexity and eight levels of collinearity we compared ways to address collinearity with standard multiple regression and machine-learning approaches. We assessed the performance of each approach by testing its impact on prediction to new data. In the extreme, we tested whether the methods were able to identify the true underlying relationship in a training dataset with strong collinearity by evaluating its performance on a test dataset without any collinearity. We found that methods specifically designed for collinearity, such as latent variable methods and tree based models, did not outperform the traditional GLM and threshold-based pre-selection. Our results highlight the value of GLM in combination with penalised methods (particularly ridge) and threshold-based pre-selection when omitted variables are considered in the final interpretation. However, all approaches tested yielded degraded predictions under change in collinearity structure and the ‘folk lore’-thresholds of correlation coefficients between predictor variables of |r| >0.7 was an appropriate indicator for when collinearity begins to severely distort model estimation and subsequent prediction. The use of ecological understanding of the system in pre-analysis variable selection and the choice of the least sensitive statistical approaches reduce the problems of collinearity, but cannot ultimately solve them.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1600-0587.2012.07348.x

Collinearity: a review of methods to deal with it and a simulation study evaluating their performance

Feedforward neural networks trained by error backpropagation are examples of nonparametric regression estimators. We present a tutorial on nonparametric inference and its relation to neural networks, and we use the statistical viewpoint to highlight strengths and weaknesses of neural models. We illustrate the main points with some recognition experiments involving artificial data as well as handwritten numerals. In way of conclusion, we suggest that current-generation feedforward neural networks are largely inadequate for difficult problems in machine perception and machine learning, regardless of parallel-versus-serial hardware or other implementation issues. Furthermore, we suggest that the fundamental challenges in neural modeling are about representation rather than learning per se. This last point is supported by additional experiments with handwritten numerals.

Neural networks and the bias/variance dilemma

Medical genetics was revolutionized during the 1980s by the application of genetic mapping to locate the genes responsible for simple Mendelian diseases. Most diseases and traits, however, do not follow simple inheritance patterns. Genetics have thus begun taking up the even greater challenge of the genetic dissection of complex traits. Four major approaches have been developed: linkage analysis, allele-sharing methods, association studies, and polygenic analysis of experimental crosses. This article synthesizes the current state of the genetic dissection of complex traits--describing the methods, limitations, and recent applications to biological problems.

https://focus.psychiatryonline.org/doi/pdf/10.1176/foc.4.3.442

Genetic dissection of complex traits

Start Analyzing a Wide Range of Problems 
Since the publication of the bestselling, highly recommended first edition, R has considerably expanded both in popularity and in the number of packages available. Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models, Second Edition takes advantage of the greater functionality now available in R and substantially revises and adds several topics.
New to the Second Edition

Expanded coverage of binary and binomial responses, including proportion responses, quasibinomial and beta regression, and applied considerations regarding these models 
New sections on Poisson models with dispersion, zero inflated count models, linear discriminant analysis, and sandwich and robust estimation for generalized linear models (GLMs) 
Revised chapters on random effects and repeated measures that reflect changes in the lme4 package and show how to perform hypothesis testing for the models using other methods
New chapter on the Bayesian analysis of mixed effect models that illustrates the use of STAN and presents the approximation method of INLA 
Revised chapter on generalized linear mixed models to reflect the much richer choice of fitting software now available
Updated coverage of splines and confidence bands in the chapter on nonparametric regression
New material on random forests for regression and classification 
Revamped R code throughout, particularly the many plots using the ggplot2 package
Revised and expanded exercises with solutions now included
Demonstrates the Interplay of Theory and Practice
This textbook continues to cover a range of techniques that grow from the linear regression model. It presents three extensions to the linear framework: GLMs, mixed effect models, and nonparametric regression models. The book explains data analysis using real examples and includes all the R commands necessary to reproduce the analyses.

Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models

Extending the linear model with R , Extending the linear model with R , کتابخانه دیجیتال جندی شاپور اهواز

Extending the Linear Model with R

Introduction Before You Start Initial Data Analysis When to Use Linear Modeling History Estimation Linear Model Matrix Representation Estimating b Least Squares Estimation Examples of Calculating b Example QR Decomposition Gauss-Markov Theorem Goodness of Fit Identifiability Orthogonality Inference Hypothesis Tests to Compare Models Testing Examples Permutation Tests Sampling Confidence Intervals for b Bootstrap Confidence Intervals Prediction Confidence Intervals for Predictions Predicting Body Fat Autoregression What Can Go Wrong with Predictions? Explanation Simple Meaning Causality Designed Experiments Observational Data Matching Covariate Adjustment Qualitative Support for Causation Diagnostics Checking Error Assumptions Finding Unusual Observations Checking the Structure of the Model Discussion Problems with the Predictors Errors in the Predictors Changes of Scale Collinearity Problems with the Error Generalized Least Squares Weighted Least Squares Testing for Lack of Fit Robust Regression Transformation Transforming the Response Transforming the Predictors Broken Stick Regression Polynomials Splines Additive Models More Complex Models Model Selection Hierarchical Models Testing-Based Procedures Criterion-Based Procedures Summary Shrinkage Methods Principal Components Partial Least Squares Ridge Regression Lasso Insurance Redlining-A Complete Example Ecological Correlation Initial Data Analysis Full Model and Diagnostics Sensitivity Analysis Discussion Missing Data Types of Missing Data Deletion Single Imputation Multiple Imputation Categorical Predictors A Two-Level Factor Factors and Quantitative Predictors Interpretation with Interaction Terms Factors with More than Two Levels Alternative Codings of Qualitative Predictors One Factor Models The Model An Example Diagnostics Pairwise Comparisons False Discovery Rate Models with Several Factors Two Factors with No Replication Two Factors with Replication Two Factors with an Interaction Larger Factorial Experiments Experiments with Blocks Randomized Block Design Latin Squares Balanced Incomplete Block Design Appendix: About R Bibliography Index

Linear Models with R

Permission to reproduce individual copies of this book for personal use is granted. Multiple copies may be created for nonprofit academic purposes — a nominal charge to cover the expense of reproduction may be made. Reproduction for profit is prohibited without permission. Preface There are many books on regression and analysis of variance. These books expect different levels of preparedness and place different emphases on the material. This book is not introductory. It presumes some knowledge of basic statistical theory and practice. Students are expected to know the essentials of statistical inference like estimation, hypothesis testing and confidence intervals. A basic knowledge of data analysis is presumed. Some linear algebra and calculus is also required. The emphasis of this text is on the practice of regression and analysis of variance. The objective is to learn what methods are available and more importantly, when they should be applied. Many examples are presented to clarify the use of the techniques and to demonstrate what conclusions can be made. There is relatively less emphasis on mathematical theory, partly because some prior knowledge is assumed and partly because the issues are better tackled elsewhere. Theory is important because it guides the approach we take. I take a wider view of statistical theory. It is not just the formal theorems. Qualitative statistical concepts are just as important in Statistics because these enable us to actually do it rather than just talk about it. These qualitative principles are harder to learn because they are difficult to state precisely but they guide the successful experienced Statistician. Data analysis cannot be learnt without actually doing it. This means using a statistical computing package. There is a wide choice of such packages. They are designed for different audiences and have different strengths and weaknesses. I have chosen to use R (ref. Ihaka and Gentleman (1996)). Why do I use R ? The are several reasons. 1. Versatility. R is a also a programming language, so I am not limited by the procedures that are preprogrammed by a package. It is relatively easy to program new methods in R. 2. Interactivity. Data analysis is inherently interactive. Some older statistical packages were designed when computing was more expensive and batch processing of computations was the norm. Despite improvements in hardware, the old batch processing paradigm lives on in their use. R does one thing at a time, allowing us to make …

Practical Regression and Anova using R

Summary. This case-study fits a variety of neural network (NN) models to the well-known airline data and compares the resulting forecasts with those obtained from the Box‐Jenkins and Holt‐ Winters methods. Many potential problems in fitting NN models were revealed such as the possibility that the fitting routine may not converge or may converge to a local minimum. Moreover it was found that an NN model which fits well may give poor out-of-sample forecasts. Thus we think it is unwise to apply NN models blindly in ‘black box’ mode as has sometimes been suggested. Rather, the wise analyst needs to use traditional modelling skills to select a good NN model, e.g. to select appropriate lagged variables as the ‘inputs’. The Bayesian information criterion is preferred to Akaike’s information criterion for comparing different models. Methods of examining the response surface implied by an NN model are examined and compared with the results of alternative nonparametric procedures using generalized additive models and projection pursuit regression. The latter imposes less structure on the model and is arguably easier to understand.

/pdf/time-series-forecasting-with-neural-networks-a-comparative-3b60emu6fj.pdf

Julian J. Faraway

Papers

Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models

Extending the Linear Model with R

Linear Models with R

Practical Regression and Anova using R

Time series forecasting with neural networks: a comparative study using the air line data