Robert W. Kennard

Bio: Robert W. Kennard is an academic researcher from University of Delaware. The author has contributed to research in topics: Polynomial regression & Mean squared error. The author has an hindex of 7, co-authored 8 publications receiving 9169 citations.

Ridge regression: biased estimation for nonorthogonal problems

Abstract: In multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory, if not incorrect, if the prediction vectors are not orthogonal. Proposed is an estimation procedure based on adding small positive quantities to the diagonal of X′X. Introduced is the ridge trace, a method for showing in two dimensions the effects of nonorthogonality. It is then shown how to augment X′X to obtain biased estimates with smaller mean square error.

Ridge Regression: Applications to Nonorthogonal Problems

Abstract: This paper is an exposition of the use of ridge regression methods. Two examples from the literature are used as a base. Attention is focused on the RIDGE TRACE which is a two-dimensional graphical procedure for portraying the complex relationships in multifactor data. Recommendations are made for obtaining a better regression equation than that given by ordinary least squares estimation.

Ridge Regression — 1980: Advances, Algorithms, and Applications

Abstract: SYNOPTIC ABSTRACTHoerl and Kennard showed in 1970 that regression coefficients estimated by their “ridge regression” methods have smaller mean squared error than those estimated by least squares. Presented here are (i) advances that have been made in extending the theory; (ii) algorithms for selecting the biasing parameter, k, that have been supported by simulation; and (iii) an annotated bibliography of the published papers, including a substantial number of applications to fields including chemical engineering, air pollution, geology, marketing, economics, and sociology among others.

A Note on a Power Generalization of Ridge Regression

Abstract: O*(k, q) = [X'X + k(X'X)-q]-'X'y = [X'X + Q'DQ]-'X'y (1.3) where D = diag (d,d2 * dp), di = k/X\i. Thus, ki = k/X,i in the general formulation of (1.1) and (1.2). In Hoerl and Kennard [1] it was shown that the optimum K matrix has elements ki = a2/ai2. Since the optimum k,'s are inversely proportional to the regression coefficients and not to the eigenvalues, it appears that exploration of the (X'X)+l variation will not prove fruitful.

The Nature of Statistical Learning Theory

Abstract: Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.

Data Mining: Practical Machine Learning Tools and Techniques

Abstract: Data Mining: Practical Machine Learning Tools and Techniques offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations. This highly anticipated third edition of the most acclaimed work on data mining and machine learning will teach you everything you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data mining. Thorough updates reflect the technical changes and modernizations that have taken place in the field since the last edition, including new material on Data Transformations, Ensemble Learning, Massive Data Sets, Multi-instance Learning, plus a new version of the popular Weka machine learning software developed by the authors. Witten, Frank, and Hall include both tried-and-true techniques of today as well as methods at the leading edge of contemporary research. *Provides a thorough grounding in machine learning concepts as well as practical advice on applying the tools and techniques to your data mining projects *Offers concrete tips and techniques for performance improvement that work by transforming the input or output in machine learning methods *Includes downloadable Weka software toolkit, a collection of machine learning algorithms for data mining tasks-in an updated, interactive interface. Algorithms in toolkit cover: data pre-processing, classification, regression, clustering, association rules, visualization

Gaussian Processes for Machine Learning

Abstract: A comprehensive and self-contained introduction to Gaussian processes, which provide a principled, practical, probabilistic approach to learning in kernel machines. Gaussian processes (GPs) provide a principled, practical, probabilistic approach to learning in kernel machines. GPs have received increased attention in the machine-learning community over the past decade, and this book provides a long-needed systematic and unified treatment of theoretical and practical aspects of GPs in machine learning. The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics. The book deals with the supervised-learning problem for both regression and classification, and includes detailed algorithms. A wide variety of covariance (kernel) functions are presented and their properties discussed. Model selection is discussed both from a Bayesian and a classical perspective. Many connections to other well-known techniques from machine learning and statistics are discussed, including support-vector machines, neural networks, splines, regularization networks, relevance vector machines and others. Theoretical issues including learning curves and the PAC-Bayesian framework are treated, and several approximation methods for learning with large datasets are discussed. The book contains illustrative examples and exercises, and code and datasets are available on the Web. Appendixes provide mathematical background and a discussion of Gaussian Markov processes.

Ridge regression: biased estimation for nonorthogonal problems

Abstract: In multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory, if not incorrect, if the prediction vectors are not orthogonal. Proposed is an estimation procedure based on adding small positive quantities to the diagonal of X′X. Introduced is the ridge trace, a method for showing in two dimensions the effects of nonorthogonality. It is then shown how to augment X′X to obtain biased estimates with smaller mean square error.

Robust enumeration of cell subsets from tissue expression profiles

Abstract: We introduce CIBERSORT, a method for characterizing cell composition of complex tissues from their gene expression profiles When applied to enumeration of hematopoietic subsets in RNA mixtures from fresh, frozen and fixed tissues, including solid tumors, CIBERSORT outperformed other methods with respect to noise, unknown mixture content and closely related cell types CIBERSORT should enable large-scale analysis of RNA mixtures for cellular biomarkers and therapeutic targets (http://cibersortstanfordedu/)