The Origins of Local Regression.- Local Regression Methods.- Fitting with LOCFIT.- Local Likelihood Estimation.- Density Estimation.- Flexible Local Regression.- Survival and Failure Time Analysis.- Discrimination and Classification.- Variance Estimation and Goodness of Fit.- Bandwidth Selection.- Adaptive Parameter Choice.- Computational Methods.- Optimizing Local Regression.

http://cyber.sibsutis.ru:82/Monarev/docs/nauka/PROBABILITY/MVsa_Statistics%20and%20applications/Loader%20C.%20Local%20Regression%20and%20Likelihood%20(Springer,%201999)(305s).pdf

Local Regression and Likelihood

Let $d$ denote a positive integer, $\|x\| = (x^2_1 + \cdots + x^2_d)^{1/2}$ the Euclidean norm of $x = (x_1, \cdots, x_d) \in \mathbb{R}^d, k$ a nonnegative integer, $\mathscr{C}_k$ the collection of $k$ times continuously differentiable functions on $\mathbb{R}^d$, and $g_k$ the Taylor polynomial of degree $k$ about the origin corresponding to $g \in \mathscr{C}_k$. Let $M$ and $p > k$ denote positive constants and let $U$ be an open neighborhood of the origin of $\mathbb{R}^d$. Let $\mathscr{G}$ denote the collection of functions $g \in \mathscr{C}_k$ such that $|g(x) - g_k(x)| \leq M \|x\|^P$ for $x\in U$. Let $m \leq k$ be a nonnegative integer, let $\theta_0\in\mathscr{C}_m$ and set $\Theta = \{\theta_0 + g:g \in \mathscr{G}\}$. Let $L$ be a linear differential operator of order $m$ on $\mathscr{C}_m$ and set $T(\theta) = L\theta(0)$ for $\theta \in \Theta$. Let $(X, Y)$ be a pair of random variables such that $X$ is $\mathbb{R}^d$ valued and $Y$ is real valued. It is assumed that the distribution of $X$ is absolutely continuous and that its density is bounded away from zero and infinity on $U$. The conditional distribution of $Y$ given $X$ is assumed to be (say) normal, with a conditional variance which is bounded away from zero and infinity on $U$. The regression function of $Y$ on $X$ is assumed to belong to $\Theta$. It is shown that $r = (p - m)/(2p + d)$ is the optimal (uniform) rate of convergence for a sequence $\{\hat{T}_n\}$ of estimators of $T(\theta)$ such that $\hat{T}_n$ is based on a random sample of size $n$ from the distribution of $(X, Y)$. An analogous result is obtained for nonparametric estimators of a density function.

/pdf/optimal-rates-of-convergence-for-nonparametric-estimators-1nxzda9fmy.pdf

Optimal Rates of Convergence for Nonparametric Estimators

We present some general results determining minimax bounds on statistical risk for density estimation based on certain information-theoretic considerations. These bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence.

/pdf/information-theoretic-determination-of-minimax-rates-of-36a7l1oz2d.pdf

Information-theoretic determination of minimax rates of convergence

We consider kernel estimation of a smooth density $f$ at a point, but depart from the usual approach in admitting an adaptive dependence of the sharpness of the kernels on the underlying density. Proportionally varying the bandwidths like $f^{-1/2}$ at the contributing readings lowers the bias to a vanishing fraction of the usual value, and makes for performance seen in well-known estimators that force moment conditions on the kernel (and so sacrifice positivity of the curve estimate). Issues of equivariance and variance stabilitization are treated.

/pdf/on-bandwidth-variation-in-kernel-estimates-a-square-root-law-2ykj7a6360.pdf

On Bandwidth Variation in Kernel Estimates-A Square Root Law

Advances in computation and the fast and cheap computational facilities now available to statisticians have had a significant impact upon statistical research, and especially the development of nonparametric data analysis procedures. In particular, theoretical and applied research on nonparametric density estimation has had a noticeable influence on related topics, such as nonparametric regression, nonparametric discrimination, and nonparametric pattern recognition. This article reviews recent developments in nonparametric density estimation and includes topics that have been omitted from review articles and books on the subject. The early density estimation methods, such as the histogram, kernel estimators, and orthogonal series estimators are still very popular, and recent research on them is described. Different types of restricted maximum likelihood density estimators, including order-restricted estimators, maximum penalized likelihood estimators, and sieve estimators, are discussed, where restrictions are imposed upon the class of densities or on the form of the likelihood function. Nonparametric density estimators that are data-adaptive and lead to locally smoothed estimators are also discussed; these include variable partition histograms, estimators based on statistically equivalent blocks, nearest-neighbor estimators, variable kernel estimators, and adaptive kernel estimators. For the multivariate case, extensions of methods of univariate density estimation are usually straightforward but can be computationally expensive. A method of multivariate density estimation that did not spring from a univariate generalization is described, namely, projection pursuit density estimation, in which both dimensionality reduction and density estimation can be pursued at the same time. Finally, some areas of related research are mentioned, such as nonparametric estimation of functionals of a density, robust parametric estimation, semiparametric models, and density estimation for censored and incomplete data, directional and spherical data, and density estimation for dependent sequences of observations.

Recent Developments in Nonparametric Density Estimation

Estimation of the value $f(0)$ of a density function evaluated at 0 is studied, $f: \mathbb{R}_m \rightarrow \mathbb{R}, 0 \in \mathbb{R}_m$. Sequences of estimators $\{\gamma_n, n \geqq 1\}$, one estimator for each sample size, are studied. We are interested in the problem, given a set $C$ of density functions and a sequence of numbers $\{a_n, n \leqq 1\}$, how rapidly can $a_n$ tend to zero and yet have $\lim\inf_{n\rightarrow\infty} \inf_{f\in C}P_f(|\gamma_n(X_1,\cdots, X_n) - f(0)|\leqq a_n) > 0?$ In brief, by "rate of convergence" we will mean the rate which $a_n$ tends to zero. For a continuum of different choices of the set $C$ specified by various Lipschitz conditions on the $k$th partial derivatives of $f, k \geqq 0$, lower bounds for the possible rate of convergence are obtained. Combination of these lower bounds with known methods of estimation give best possible rates of convergence in a number of cases.

/pdf/on-the-best-obtainable-asymptotic-rates-of-convergence-in-4sp2wvsfaj.pdf

R. H. Farrell

Papers

On the Best Obtainable Asymptotic Rates of Convergence in Estimation of a Density Function at a Point