Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

Machine Learning : A Probabilistic Perspective

OBJECTIVE Statistical models, such as linear or logistic regression or survival analysis, are frequently used as a means to answer scientific questions in psychosomatic research. Many who use these techniques, however, apparently fail to appreciate fully the problem of overfitting, ie, capitalizing on the idiosyncrasies of the sample at hand. Overfitted models will fail to replicate in future samples, thus creating considerable uncertainty about the scientific merit of the finding. The present article is a nontechnical discussion of the concept of overfitting and is intended to be accessible to readers with varying levels of statistical expertise. The notion of overfitting is presented in terms of asking too much from the available data. Given a certain number of observations in a data set, there is an upper limit to the complexity of the model that can be derived with any acceptable degree of uncertainty. Complexity arises as a function of the number of degrees of freedom expended (the number of predictors including complex terms such as interactions and nonlinear terms) against the same data set during any stage of the data analysis. Theoretical and empirical evidence--with a special focus on the results of computer simulation studies--is presented to demonstrate the practical consequences of overfitting with respect to scientific inference. Three common practices--automated variable selection, pretesting of candidate predictors, and dichotomization of continuous variables--are shown to pose a considerable risk for spurious findings in models. The dilemma between overfitting and exploring candidate confounders is also discussed. Alternative means of guarding against overfitting are discussed, including variable aggregation and the fixing of coefficients a priori. Techniques that account and correct for complexity, including shrinkage and penalization, also are introduced.

/pdf/what-you-see-may-not-be-what-you-get-a-brief-nontechnical-3gj4oye3vz.pdf

What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models.

The generalized estimating equation (GEE) approach of Zeger and Liang facilitates analysis of data collected in longitudinal, nested, or repeated measures designs. GEEs use the generalized linear model to estimate more efficient and unbiased regression parameters relative to ordinary least squares regression in part because they permit specification of a working correlation matrix that accounts for the form of within-subject correlation of responses on dependent variables of many different distributions, including normal, binomial, and Poisson. The author briefly explains the theory behind GEEs and their beneficial statistical properties and limitations and compares GEEs to suboptimal approaches for analyzing longitudinal data through use of two examples. The first demonstration applies GEEs to the analysis of data from a longitudinal lab study with a counted response variable; the second demonstration applies GEEs to analysis of data with a normally distributed response variable from subjects nested with...

Using Generalized Estimating Equations for Longitudinal Data Analysis

Background— Late stent thrombosis (LST) after Cypher and Taxus drug-eluting stent placement has emerged as a major concern. Although the clinical predictors of LST have been reported, specific morphological and histological correlates of LST remain unknown. Methods and Results— From a registry totaling 81 human autopsies of drug-eluting stents, 46 (62 lesions) had a drug-eluting stent implanted >30 days. We identified 28 lesions with thrombus and compared those with 34 of similar duration without thrombosis using computer-guided morphometric and histological analyses. LST was defined as an acute thrombus within a coronary artery stent in place >30 days. Multiple logistic generalized estimating equations modeling demonstrated that endothelialization was the best predictor of thrombosis. The morphometric parameter that best correlated with endothelialization was the ratio of uncovered to total stent struts per section. A univariable logistic generalized estimating equations model of occurrence of thrombus i...

Pathological Correlates of Late Drug-Eluting Stent Thrombosis Strut Coverage as a Marker of Endothelialization

Despite ubiquitous references to Pfeffer and Salancik's classic volume, The External Control of Organizations, resource dependence theory is more of an appealing metaphor than a foundation for test...

Power Imbalance, Mutual Dependence, and Constraint Absorption: A Closer Look at Resource Dependence Theory

Christensen, R. (2002), Plane Answers to Complex Questions: The Theory of Linear Models (3rd ed.), New York: Springer-Verlag. Crocker, D. C. (1980), Review of Linear Regression Analysis, by G. A. F. Seber, Technometrics, 22, 130. Datta, B. N. (1995), Numerical Linear Algebra and Applications, Paci c Grove, CA: Brooks/Cole. Draper, N. R. (2002), “Applied Regression Analysis Bibliography Update 2000–2001,” Communications in Statistics: Theory and Methods, 2051– 2075. Golub, G. H., and Van Loan, C. F. (1996), Matrix Computations (3rd ed.), Baltimore, MD: Johns Hopkins University Press. Graybill, F. A. (2000), Theory and Application of the Linear Model, Paci c Grove, CA: Brooks/Cole. Hocking, R. R. (2003), Methods and Applications of Linear Models: Regression and the Analysis of Variance (2nd ed.), New York: Wiley. Porat, B. (1993), Digital Processing of Random Signals, Englewood Cliffs, NJ: Prentice-Hall. Ravishanker, N., and Dey, D. K. (2002), A First Course in Linear Model Theory, Boca Raton, FL: Chapman and Hall/CRC. White, H. (1984), Asymptotic Theory for Econometricians, Orlando, FL: Academic Press.

Edward C Chao

Papers

Generalized Estimating Equations