Function estimation/approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansions and steepest-descent minimization. A general gradient descent “boosting” paradigm is developed for additive expansions based on any fitting criterion.Specific algorithms are presented for least-squares, least absolute deviation, and Huber-M loss functions for regression, and multiclass logistic likelihood for classification. Special enhancements are derived for the particular case where the individual additive components are regression trees, and tools for interpreting such “TreeBoost” models are presented. Gradient boosting of regression trees produces competitive, highly robust, interpretable procedures for both regression and classification, especially appropriate for mining less than clean data. Connections between this approach and the boosting methods of Freund and Shapire and Friedman, Hastie and Tibshirani are discussed.

/pdf/greedy-function-approximation-a-gradient-boosting-machine-13vts2e34v.pdf

Greedy function approximation: A gradient boosting machine.

/pdf/neural-networks-and-deep-learning-3ccs3ain8p.pdf

Neural Networks and Deep Learning

/pdf/neural-network-toolbox-user-s-guide-ll1bn7nko0.pdf

Neural Network Toolbox™ User's Guide

Aggregation of information is of primary importance in the construction of knowledge based systems in various domains, ranging from medicine, economics, and engineering to decision-making processes, artificial intelligence, robotics, and machine learning. This book gives a broad introduction into the topic of aggregation functions, and provides a concise account of the properties and the main classes of such functions, including classical means, medians, ordered weighted averaging functions, Choquet and Sugeno integrals, triangular norms, conorms and copulas, uninorms, nullnorms, and symmetric sums. It also presents some state-of-the-art techniques, many graphical illustrations and new interpolatory aggregation functions. A particular attention is paid to identification and construction of aggregation functions from application specific requirements and empirical data. This book provides scientists, IT specialists and system architects with a self-contained easy-to-use guide, as well as examples of computer code and a software package. It will facilitate construction of decision support, expert, recommender, control and many other intelligent systems.

Aggregation Functions: A Guide for Practitioners

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.

/pdf/deformable-medical-image-registration-a-survey-2uee59wp1z.pdf

Deformable Medical Image Registration: A Survey

Preface 1. Introduction 2. Summary of methods and applications 3. General methods for approximation and interpolation 4. Radial basis function approximation on infinite grids 5. Radial basis functions on scattered data 6. Radial basis functions with compact support 7. Implementations 8. Least squares methods 9. Wavelet methods with radial basis functions 10. Further results and open problems Appendix Bibliography Index.

/pdf/radial-basis-functions-theory-and-implementations-2a855oz1nc.pdf

Radial Basis Functions: Theory and Implementations

From the Publisher:
"In many areas of mathematics, science and engineering, from computer graphics to inverse methods to signal processing it is necessary to estimate parameters, usually multidimensional, by approximation and interpolation. Radial basis functions are a modern and powerful tool which work well in very general circumstances, and so are becoming of widespread use, as the limitations of other methods, such as least squares, polynomial interpolation or wavelet-based, become apparent." This is the first book devoted to the subject and the author's aim is to give a thorough treatment from both the theoretical and practical implementation viewpoints. For example, he emphasises the many positive features of radial basis functions such as the unique solvability of the interpolation problem, the computation of interpolants, their smoothness and convergence, and provides a careful classification of the radial basis functions into types that have different convergence. A comprehensive bibliography rounds off what will prove a very valuable work.

/pdf/radial-basis-functions-10d8ilue5x.pdf

Radial Basis Functions

Radial basis functions are well-known and successful tools for the interpolation of data in many dimensions. Several radial basis functions of compact support that give rise to nonsingular interpolation problems have been proposed, and in this paper we study a new, larger class of smooth radial functions of compact support which contains other compactly supported ones that were proposed earlier in the literature.

/pdf/a-new-class-of-radial-basis-functions-with-compact-support-5ed951xtw6.pdf

A new class of radial basis functions with compact support

In this paper we study discretizations of the general pantograph equation y'(t) = ay(t) + by(6(t)) + cy'(O(t)), t > 0, y(O) = Yo, where a, b, c, and yo are complex numbers and where 0 and 0 are strictly increasing functions on the nonnegative reals with 6(0) = q(O) = 0 and 0(t) < t, 0(t) < t for positive t. Our purpose is an analysis of the stability of the numerical solution with trapezoidal rule discretizations, and we will identify conditions on a, b, c and the stepsize which imply that the solution sequence {yn }I??= is bounded or that it tends to zero algebraically, as a negative power of n.

/pdf/stability-of-the-discretized-pantograph-differential-5dfbtzl84h.pdf

Stability of the discretized pantograph differential equation

In this paper, radial basis functions that are compactly supported and give rise to positive definite interpolation matrices for scattered data are discussed. They are related to the well-known thin plate spline radial functions which are highly useful in applications for gridfree approximation methods. Also, encouraging approximation results for the compactly supported radial functions are shown.

/pdf/radial-functions-on-compact-support-5g659blnlt.pdf

Martin D. Buhmann

Papers

Radial Basis Functions: Theory and Implementations

Radial Basis Functions

A new class of radial basis functions with compact support

Stability of the discretized pantograph differential equation

Radial functions on compact support