On robust estimation of the location parameter

Why is Nonparametric Regression Important? * How to Construct Nonparametric Regression Estimates * Lower Bounds * Partitioning Estimates * Kernel Estimates * k-NN Estimates * Splitting the Sample * Cross Validation * Uniform Laws of Large Numbers * Least Squares Estimates I: Consistency * Least Squares Estimates II: Rate of Convergence * Least Squares Estimates III: Complexity Regularization * Consistency of Data-Dependent Partitioning Estimates * Univariate Least Squares Spline Estimates * Multivariate Least Squares Spline Estimates * Neural Networks Estimates * Radial Basis Function Networks * Orthogonal Series Estimates * Advanced Techniques from Empirical Process Theory * Penalized Least Squares Estimates I: Consistency * Penalized Least Squares Estimates II: Rate of Convergence * Dimension Reduction Techniques * Strong Consistency of Local Averaging Estimates * Semi-Recursive Estimates * Recursive Estimates * Censored Observations * Dependent Observations

http://www.gbv.de/dms/hebis-darmstadt/toc/106925350.pdf

A Distribution-Free Theory of Nonparametric Regression

We combine ideas from machine learning (ML) and operations research and management science (OR/MS) in developing a framework, along with specific methods, for using data to prescribe optimal decisi...

From Predictive to Prescriptive Analytics

/pdf/image-analysis-by-moments-reconstruction-and-computational-26nmij4inl.pdf

Image analysis by moments : reconstruction and computational aspects

Let {X/sub t/} be a real-valued time series. The best nonlinear predictor of X/sub 0/ given the infinite past X/sub -/spl infin///sup -1/ in the least squares sense, is equal to the conditional mean E{X/sub 0/|X/sub -/spl infin///sup -1/}. Previously, it has been shown that certain predictors based on growing segments of past observations converge to the best predictor given the infinite past whenever {X/sub t/} is a stationary process with values in a bounded interval. The present paper deals with universal prediction schemes for stationary processes with finite mean. We also discuss universal schemes for learning the conditional mean E{X/sub 0/|X/sub -/spl infin///sup -1/Y/sub -/spl infin///sup -1/Y/sub 0/} from past observations of a stationary pair process {(X/sub t/, Y/sub t/)}, and schemes for learning the repression function m(y)=E{X|Y=y} from independent samples of (X, Y).

Universal schemes for learning the best nonlinear predictor given the infinite past and side information

A number of generalizations of the Kolmogorov strong law of large numbers are known including convex combinations of random variables (rvs) with random coefficients. In the case of pairs of i.i.d. rvs ( X 1,Y 1),...,(X n,Y n) , with μ being the probability distribution of the x s, the averages of the Y s for which the accompanying X s are in a vicinity of a given point x may converge with probability 1 (w.p. 1) and for μ -almost everywhere ( μ a.e.) x to conditional expectation r (x)=E(Y|X=x) . We consider the Nadaraya-Watson estimator of E (Y|X=x) where the vicinities of x are determined by window widths h n . Its convergence towards r (x) w.p. 1 and for μ a.e. x under the condition E |Y|<∞ is called a strong law of large numbers for conditional expectations (SLLNCE). If no other assumptions on μ except that implied by E |Y|<∞ are required then the SLLNCE is called universal. In the present paper we investigate the minimal assumptions for the SLLNCE and for the universal SLLNCE. We improve the best-known results in this direction.

/pdf/on-a-universal-strong-law-of-large-numbers-for-conditional-ai42gzsmh2.pdf

On a universal strong law of large numbers for conditional expectations

A rule of thumb (not only) for gamblers

This paper explores a class of robust estimators of normal quantiles filling the gap between maximum likelihood estimators and empirical quantiles. Our estimators are linear combinations of M-estimators. Their asymptotic variances can be arbitrarily close to variances of the maximum likelihood estimators. Compared with empirical quantiles, the new estimators offer considerable reduction of variance at near normal probability distributions.

https://projecteuclid.org/download/pdf_1/euclid.aos/1059655910

On M-estimators and normal quantiles

Let X1, . . ., Xn be independent identically distributed random variables with a common continuous (cumulative) distribution function (d.f.) F, and F^n the empirical d.f. (e.d.f.) based on X1, . . ., Xn. Let G be a smooth d.f. and Gθ = G (·–θ) its translation through θ∈R. Using a Kolmogorov-Levy type metric ρα defined on the space of d.f.s. on R, the paper derives both null and non-null limiting distributions of √n[ρα(Fn, Gθn) –ρα (F, Gθ)], √n(θn –θ) and √nρα (Gθ, Gθ), where θn and θ are the minimum ρα-distance parameters for Fn and F from G, respectively. These distributions are known explicitly in important particular cases; with some complementary Monte Carlo simulations, they help us clarify our understanding of estimation using minimum distance methods and supremum type metrics. We advocate use of the minimum distance method with supremum type metrics in cases of non-null models. The resulting functionals are Hadamard differentiable and efficient. For small scale parameters the minimum distance functionals are close to medians of the parent distributions. The optimal small scale models result in minimum distance estimators having asymptotic variances very competitive and comparable with best known robust estimators.

Andrzej S. Kozek

Papers

On a universal strong law of large numbers for conditional expectations

A rule of thumb (not only) for gamblers

On M-estimators and normal quantiles

On minimum distance estimation using Kolmogorov-Lévy type metrics

Distribution-free consistency of kernel non-parametric M-estimators