Département de Mathématiques

About: Département de Mathématiques is a facility organization based out in Orsay, France. It is known for research contribution in the topics: Estimator & Computer science. The organization has 526 authors who have published 1192 publications receiving 14018 citations. The organization is also known as: Laboratoire de Mathématiques d’Orsay.

VSURF: An R Package for Variable Selection Using Random Forests

Abstract: This paper describes the R package VSURF. Based on random forests, and for both regression and classification problems, it returns two subsets of variables. The first is a subset of important variables including some redundancy which can be relevant for interpretation, and the second one is a smaller subset corresponding to a model trying to avoid redundancy focusing more closely on prediction objective. The two-stage strategy is based on a preliminary ranking of the explanatory variables using the random forests permutation-based score of importance and proceeds using a stepwise forward strategy for variable introduction. The two proposals can be obtained automatically using data-driven default values, good enough to provide interesting results, but can also be tuned by the user. The algorithm is illustrated on a simulated example and its applications to real datasets are presented.

On Necessary and Sufficient Conditions for LP-Estimates of Riesz Transforms Associated to Elliptic Operators on RN and Related Estimates

Abstract: Beyond Calderon-Zygmund operators Basic $L^2$ theory for elliptic operators $L^p$ theory for the semigroup $L^p$ theory for square roots Riesz transforms and functional calculi Square function estimates Miscellani Appendix A. Calderon-Zygmund decomposition for Sobolev functions Appendix. Bibliography.

Weak Dependence: With Examples and Applications

Abstract: This monograph is aimed at developing Doukhan/Louhichi's (1999) idea to measure asymptotic independence of a random process. The authors propose various examples of models fitting such conditions such as stable Markov chains, dynamical systems or more complicated models, nonlinear, non-Markovian, and heteroskedastic models with infinite memory. Most of the commonly used stationary models fit their conditions. The simplicity of the conditions is also their strength. The main existing tools for an asymptotic theory are developed under weak dependence. They apply the theory to nonparametric statistics, spectral analysis, econometrics, and resampling. The level of generality makes those techniques quite robust with respect to the model. The limit theorems are sometimes sharp and always simple to apply. The theory (with proofs) is developed and the authors propose to fix the notation for future applications. A large number of research papers deals with the present ideas; the authors as well as numerous other investigators participated actively in the development of this theory. Several applications are still needed to develop a method of analysis for (nonlinear) times series and they provide here a strong basis for such studies. Jerome Dedecker (associate professor Paris 6), Gabriel Lang (professor at Ecole Polytechnique, ENGREF Paris), Sana Louhichi (Paris 11, associate professor at Paris 2), and Clementine Prieur (associate professor at INSA, Toulouse) are main contributors for the development of weak dependence. Jose Rafael Leon (Polar price, correspondent of the Bernoulli society for Latino-America) is professor at University Central of Venezuela and Paul Doukhan is professor at ENSAE (SAMOS-CES Paris 1 and Cergy Pontoise) and associate editor of Stochastic Processes and their Applications. His Mixing: Properties and Examples (Springer, 1994) is a main reference for the concurrent notion of mixing.

Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss

Abstract: Neural networks trained to minimize the logistic (a.k.a. cross-entropy) loss with gradient-based methods are observed to perform well in many supervised classification tasks. Towards understanding this phenomenon, we analyze the training and generalization behavior of infinitely wide two-layer neural networks with homogeneous activations. We show that the limits of the gradient flow on exponentially tailed losses can be fully characterized as a max-margin classifier in a certain non-Hilbertian space of functions. In presence of hidden low-dimensional structures, the resulting margin is independent of the ambiant dimension, which leads to strong generalization bounds. In contrast, training only the output layer implicitly solves a kernel support vector machine, which a priori does not enjoy such an adaptivity. Our analysis of training is non-quantitative in terms of running time but we prove computational guarantees in simplified settings by showing equivalences with online mirror descent. Finally, numerical experiments suggest that our analysis describes well the practical behavior of two-layer neural networks with ReLU activation and confirm the statistical benefits of this implicit bias.

Data-driven Calibration of Penalties for Least-Squares Regression

Abstract: Penalization procedures often suffer from their dependence on multiplying factors, whose optimal values are either unknown or hard to estimate from data. We propose a completely data-driven calibration algorithm for these parameters in the least-squares regression framework, without assuming a particular shape for the penalty. Our algorithm relies on the concept of minimal penalty, recently introduced by Birge and Massart (2007) in the context of penalized least squares for Gaussian homoscedastic regression. On the positive side, the minimal penalty can be evaluated from the data themselves, leading to a data-driven estimation of an optimal penalty which can be used in practice; on the negative side, their approach heavily relies on the homoscedastic Gaussian nature of their stochastic framework. The purpose of this paper is twofold: stating a more general heuristics for designing a data-driven penalty (the slope heuristics) and proving that it works for penalized least-squares regression with a random design, even for heteroscedastic non-Gaussian data. For technical reasons, some exact mathematical results will be proved only for regressogram bin-width selection. This is at least a first step towards further results, since the approach and the method that we use are indeed general.