Matrix Differential Calculus with Applications in Statistics and Econometrics

Recent years have witnessed an explosion in the volume and variety of data collected in all scientific disciplines and industrial settings. Such massive data sets present a number of challenges to researchers in statistics and machine learning. This book provides a self-contained introduction to the area of high-dimensional statistics, aimed at the first-year graduate level. It includes chapters that are focused on core methodology and theory - including tail bounds, concentration inequalities, uniform laws and empirical process, and random matrices - as well as chapters devoted to in-depth exploration of particular model classes - including sparse linear models, matrix models with rank constraints, graphical models, and various types of non-parametric models. With hundreds of worked examples and exercises, this text is intended both for courses and for self-study by graduate students and researchers in statistics, machine learning, and related fields who must understand, apply, and adapt modern statistical methods suited to large-scale data.

High-Dimensional Statistics: A Non-Asymptotic Viewpoint

Concrete mathematics, a foundation for computer science, by Ronald L. Graham, Donald E. Knuth and Oren Patashnik. Pp 625. £24·95. 1989. ISBN 0-201-14236-8 (Addison-Wesley)

Diffusion processes and their sample paths

1. Nonparametric statistical models 2. Gaussian processes 3. Empirical processes 4. Function spaces and approximation theory 5. Linear nonparametric estimators 6. The minimax paradigm 7. Likelihood-based procedures 8. Adaptive inference.

Mathematical foundations of infinite-dimensional statistical models

We study nonparametric maximum likelihood estimation of a log-concave probability density and its distribution and hazard function. Some general properties of these estimators are derived from two characterizations. It is shown that the rate of convergence with respect to supremum norm on a compact interval for the density and hazard rate estimator is at least $(\log(n)/n)^{1/3}$ and typically $(\log(n)/n)^{2/5}$, whereas the difference between the empirical and estimated distribution function vanishes with rate $o_{\mathrm{p}}(n^{-1/2})$ under certain regularity assumptions.

Maximum likelihood estimation of a log-concave density and its distribution function: Basic properties and uniform consistency

Background Many preschool children have wheeze or cough, but only some have asthma later. Existing prediction tools are difficult to apply in clinical practice or exhibit methodological weaknesses. Objective We sought to develop a simple and robust tool for predicting asthma at school age in preschool children with wheeze or cough. Methods From a population-based cohort in Leicestershire, United Kingdom, we included 1- to 3-year-old subjects seeing a doctor for wheeze or cough and assessed the prevalence of asthma 5 years later. We considered only noninvasive predictors that are easy to assess in primary care: demographic and perinatal data, eczema, upper and lower respiratory tract symptoms, and family history of atopy. We developed a model using logistic regression, avoided overfitting with the least absolute shrinkage and selection operator penalty, and then simplified it to a practical tool. We performed internal validation and assessed its predictive performance using the scaled Brier score and the area under the receiver operating characteristic curve. Results Of 1226 symptomatic children with follow-up information, 345 (28%) had asthma 5 years later. The tool consists of 10 predictors yielding a total score between 0 and 15: sex, age, wheeze without colds, wheeze frequency, activity disturbance, shortness of breath, exercise-related and aeroallergen-related wheeze/cough, eczema, and parental history of asthma/bronchitis. The scaled Brier scores for the internally validated model and tool were 0.20 and 0.16, and the areas under the receiver operating characteristic curves were 0.76 and 0.74, respectively. Conclusion This tool represents a simple, low-cost, and noninvasive method to predict the risk of later asthma in symptomatic preschool children, which is ready to be tested in other populations.

/pdf/a-simple-asthma-prediction-tool-for-preschool-children-with-yvrr80inj3.pdf

A simple asthma prediction tool for preschool children with wheeze or cough

We introduce a multiscale test statistic based on local order statistics and spacings that provides simultaneous confidence statements for the existence and location of local increases and decreases of a density or a failure rate. The procedure provides guaranteed finite-sample significance levels, is easy to implement and possesses certain asymptotic optimality and adaptivity properties.

Multiscale inference about a density

We study the approximation of arbitrary distributions $P$ on $d$-dimensional space by distributions with log-concave density. Approximation means minimizing a Kullback--Leibler-type functional. We show that such an approximation exists if and only if $P$ has finite first moments and is not supported by some hyperplane. Furthermore we show that this approximation depends continuously on $P$ with respect to Mallows distance $D_1(\cdot,\cdot)$. This result implies consistency of the maximum likelihood estimator of a log-concave density under fairly general conditions. It also allows us to prove existence and consistency of estimators in regression models with a response $Y=\mu(X)+\epsilon$, where $X$ and $\epsilon$ are independent, $\mu(\cdot)$ belongs to a certain class of regression functions while $\epsilon$ is a random error with log-concave density and mean zero.

Approximation by log-concave distributions, with applications to regression

We develop an active set algorithm for the maximum likelihood estimation of a log-concave density based on complete data. Building on this fast algorithm, we indidate an EM algorithm to treat arbitrarily censored or binned data.

Lutz Duembgen

Papers

Maximum likelihood estimation of a log-concave density and its distribution function: Basic properties and uniform consistency

A simple asthma prediction tool for preschool children with wheeze or cough

Multiscale inference about a density

Approximation by log-concave distributions, with applications to regression

Active Set and EM Algorithms for Log-Concave Densities Based on Complete and Censored Data