Matrix Differential Calculus with Applications in Statistics and Econometrics

Recent years have witnessed an explosion in the volume and variety of data collected in all scientific disciplines and industrial settings. Such massive data sets present a number of challenges to researchers in statistics and machine learning. This book provides a self-contained introduction to the area of high-dimensional statistics, aimed at the first-year graduate level. It includes chapters that are focused on core methodology and theory - including tail bounds, concentration inequalities, uniform laws and empirical process, and random matrices - as well as chapters devoted to in-depth exploration of particular model classes - including sparse linear models, matrix models with rank constraints, graphical models, and various types of non-parametric models. With hundreds of worked examples and exercises, this text is intended both for courses and for self-study by graduate students and researchers in statistics, machine learning, and related fields who must understand, apply, and adapt modern statistical methods suited to large-scale data.

High-Dimensional Statistics: A Non-Asymptotic Viewpoint

Concrete mathematics, a foundation for computer science, by Ronald L. Graham, Donald E. Knuth and Oren Patashnik. Pp 625. £24·95. 1989. ISBN 0-201-14236-8 (Addison-Wesley)

Diffusion processes and their sample paths

1. Nonparametric statistical models 2. Gaussian processes 3. Empirical processes 4. Function spaces and approximation theory 5. Linear nonparametric estimators 6. The minimax paradigm 7. Likelihood-based procedures 8. Adaptive inference.

Mathematical foundations of infinite-dimensional statistical models

We derive limiting distributions of symmetrized estimators of scatter, where instead of all $n(n-1)/2$ pairs of the $n$ observations we only consider $nd$ suitably chosen pairs, $1 \le d<\lfloor n/2\rfloor$. It turns out that the resulting estimators are asymptotically equivalent to the original one whenever $d = d(n) \to \infty$ at arbitrarily slow speed. We also investigate the asymptotic properties for arbitrary fixed $d$. These considerations and numerical examples indicate that for practical purposes, moderate fixed values of $d$ between,say, $10$ and $20$ yield already estimators which are computationally feasible and rather close to the original ones.

Approximating Symmetrized Estimators of Scatter via Balanced Incomplete U-Statistics

Consider the standard linear regression model $Y = D \theta + \epsilon$ with given design matrix $D$ ($n \times p$), unknown parameter $\theta$ ($p \times 1$) and unobserved error vector $\epsilon$ ($n \times 1$) with i.i.d.\ centered Gaussian components. Motivated by an application in economics, we compare the ranks $R_i$ of the errors $\epsilon_i$ with the ranks $\hat{R}_i$ of the residuals $\hat{\epsilon}_i$, where $\hat{\epsilon} = Y - D \hat{\theta}$ with the least squares estimator $\hat{\theta}$. Exact and approximate formulae are given for the rank distortions $\sqrt{E \, (\hat{R}_i - R_i)^2}$.

On Ranks of Regression Errors and Residuals

We derive multiscale statistics for deconvolution in order to detect qualitative features of the unknown density. An important example covered within this framework is to test for local monotonicity on all scales simultaneously. We investigate the moderately ill-posed setting, where the Fourier transform of the error density in the deconvolution model is of polynomial decay. For multiscale testing, we consider a calibration, motivated by the modulus of continuity of Brownian motion. We investigate the performance of our results from both the theoretical and simulation based point of view. A major consequence of our work is that the detection of qualitative features of a density in a deconvolution problem is a doable task although the minimax rates for pointwise estimation are very slow.

Multiscale Methods for Shape Constraints in Deconvolution

. We provide conﬁdence bands for isotonic quantile curves in nonparametric univariate regression with guaranteed given coverage probability . The method is an adaptation of the conﬁdence bands of D¨umbgen and Johns (2004) for isotonic median curves.

Honest Confidence Bands for Isotonic Quantile Curves

The usual stochastic order and the likelihood ratio order between probability distributions on the real line are reviewed in full generality. In addition, for the distribution of a random pair (X,Y), it is shown that the conditional distributions of Y, given X = x, are increasing in x with respect to the likelihood ratio order if and only if the joint distribution of (X,Y) is totally positive of order two (TP2) in a certain sense. It is also shown that these three types of constraints are stable under weak convergence, and that weak convergence of TP2 distributions implies convergence of the conditional distributions just mentioned.

/pdf/on-stochastic-orders-and-total-positivity-3n96udui.pdf

Lutz Duembgen

Papers

Approximating Symmetrized Estimators of Scatter via Balanced Incomplete U-Statistics

On Ranks of Regression Errors and Residuals

Multiscale Methods for Shape Constraints in Deconvolution

Honest Confidence Bands for Isotonic Quantile Curves

On stochastic orders and total positivity