In the last ten years, there has been increasing interest and activity in the general area of partially linear regression smoothing in statistics. Many methods and techniques have been proposed and studied. This monograph hopes to bring an up-to-date presentation of the state of the art of partially linear regression techniques. The emphasis of this monograph is on methodologies rather than on the theory, with a particular focus on applications of partially linear regression techniques to various statistical problems. These problems include least squares regression, asymptotically efficient estimation, bootstrap resampling, censored data analysis, linear measurement error models, nonlinear measurement models, nonlinear and nonparametric time series models.
We hope that this monograph will serve as a useful reference for theoretical and applied statisticians and to graduate students and others who are interested in the area of partially linear regression. While advanced mathematical ideas have been valuable in some of the theoretical development, the methodological power of partially linear regression can be demonstrated and discussed without advanced mathematics.
This monograph can be divided into three parts: part one–Chapter 1 through Chapter 4; part two–Chapter 5; and part three–Chapter 6. In the first part, we discuss various estimators for partially linear regression models, establish theo- retical results for the estimators, propose estimation procedures, and implement the proposed estimation procedures through real and simulated examples.
The second part is of more theoretical interest. In this part, we construct several adaptive and efficient estimates for the parametric component. We show that the LS estimator of the parametric component can be modified to have both Bahadur asymptotic efficiency and second order asymptotic efficiency. In the third part, we consider partially linear time series models. First, we propose a test procedure to determine whether a partially linear model can be used to fit a given set of data. Asymptotic test criteria and power investigations are presented. Second, we propose a Cross-Validation (CV) based criterion to select the optimum linear subset from a partially linear regression and estab- lish a CV selection criterion for the bandwidth involved in the nonparametric kernel estimation. The CV selection criterion can be applied to the case where the observations fitted by the partially linear model (1.1.1) are independent and identically distributed (i.i.d.). Due to this reason, we have not provided a sepa- rate chapter to discuss the selection problem for the i.i.d. case. Third, we provide recent developments in nonparametric and semiparametric time series regression.
This work of the authors was supported partially by the Sonderforschungs- bereich373“QuantifikationundSimulationO konomischerProzesse”.Thesecond author was also supported by the National Natural Science Foundation of China and an Alexander von Humboldt Fellowship at the Humboldt University, while the third author was also supported by the Australian Research Council. The second and third authors would like to thank their teachers: Professors Raymond Car- roll, Guijing Chen, Xiru Chen, Ping Cheng and Lincheng Zhao for their valuable inspiration on the two authors’ research efforts. We would like to express our sin- cere thanks to our colleagues and collaborators for many helpful discussions and stimulating collaborations, in particular, Vo Anh, Shengyan Hong, Enno Mam- men, Howell Tong, Axel Werwatz and Rodney Wolff. For various ways in which they helped us, we would like to thank Adrian Baddeley, Rong Chen, Anthony Pettitt, Maxwell King, Michael Schimek, George Seber, Alastair Scott, Naisyin Wang, Qiwei Yao, Lijian Yang and Lixing Zhu.
The authors are grateful to everyone who has encouraged and supported us to finish this undertaking. Any remaining errors are ours.

/pdf/partially-linear-models-gdlfizwk9w.pdf

Partially Linear Models

Geometrical foundations of asymptotic inference are described in simple cases, without the machinery of differential geometry. A primary statistical goal is to provide a deeper understanding of the ideas of Fisher and Jeffreys. The role of differential geometry in generalizing results is indicated, further applications are mentioned, and geometrical methods in nonlinear regression are related to those developed for general parametric families.

/pdf/the-geometry-of-asymptotic-inference-3556br1p6s.pdf

The Geometry of Asymptotic Inference

Modern Probability Theory and Its Applications

Recent developments in empirical likelihood (EL) methods are reviewed. First, to put the method inperspective, two interpretations of empirical likelihood are presented, one as a nonparametric maximum likelihood estimation method (NPMLE) and the other as a generalized minimum contrast estimator(GMC). The latter interpretation provides a clear connection between EL, GMM, GEL and other related estimators. Second, EL is shown to have various advantages over other methods. The theory of large deviations demonstrates that EL emerges naturally in achieving asymptotic optimality both for estimation and testing. Interestingly, higher order asymptotic analysis also suggests that EL is generally a preferred method. Third, extensions of EL are discussed in various settings, including estimation of conditional moment restriction models, nonparametric specification testing and time series models. Finally, practical issues in applying EL to real data, such as computational algorithms for EL, are discussed. Numerical examples to illustrate the efficacy of the method are presented.

Empirical Likelihood Methods in Econometrics: Theory and Practice

Following Jureckova (1981) we introduce a finite-sample measure of performance of regression estimators based on tail behavior. The least squares estimator is studied in detail, and we find that it achieves good tail performance under strictly Gaussian conditions. However, the tail performance of the least-squares estimator is found to be extremely poor in the case of heavy-tailed error distributions. Further analysis of the least-squares estimator with light-tailed errors indicates the strong influence of the design matrix in determining tail performance. Turning to the tail behavior of various robust estimators of the parameters of the linear model, we focus on tail performance under heavy (algebraic) tailed errors. The 11-estimator is seen to be a leading case: we find a simple characterization of its tail behavior in terms of the design configuration and show that a broad class of M-estimators have the same performance. Perhaps most significantly, it is shown that our finite-sample measure of tail performance is, for heavy tailed error distributions, essentially the same as the finite sample concept of breakdown point introduced by Donoho and Huber (1983). This finding provides an important probabilistic interpretation of the breakdown point and clarifies the role of tail behavior as a quantitative measure of robustness. This link is further explored for high-breakdown regression estimators including Rousseeuw's (1984) leastmedian-of-squares estimator.

/pdf/tail-behavior-of-regression-estimators-and-their-breakdown-4ztv0oymg8.pdf

Tail behavior of regression estimators and their breakdown points

In this paper the exponential rates of decrease and bounds on tail probabilities for consistent estimators are studied using large deviation methods. The asymptotic expansions of Bahadur bounds and exponential rates in the case of the maximum likelihood estimator are obtained. Based on these results we have obtained a result parallel to the Fisher-Rao-Efron result concerning second-order efficiency (see Efron, 1975). Our results also substantiate the geometric observation given by Efron (1975) that if the statistical curvature of the underlying distribution is small, then the maximum likelihood estimator is nearly optimal.

/pdf/large-sample-point-estimation-a-large-deviation-theory-559n470gkb.pdf

Large Sample Point Estimation: A Large Deviation Theory Approach

In this paper, we have proved a fundamental property of the characteristic function for the random variable $(\partial/\partial\theta) \log f(x \mid \theta)$. Based on this result, we have proved under regularity conditions different from Bahadur's that certain classes of consistent estimators $\{\theta_n^\ast\}$ are asymptotically efficient in Bahadur's sense $\lim_{\varepsilon \rightarrow 0} \lim_{n \rightarrow \infty} \frac{1}{n\varepsilon^2} \log P\theta\{|\theta_n^\ast - \theta| \geqq \varepsilon\} = -\frac{I(\theta)}{2}.$ Our proof also gives a simple and direct method to verify Bahadur's [2] result.

/pdf/on-a-theorem-of-bahadur-on-the-rate-of-convergence-of-point-24364sgpvy.pdf

On a Theorem of Bahadur on the Rate of Convergence of Point Estimators

The rate at which the probability $P_\theta\{|t_n - \theta| \geqq \varepsilon\}$ of consistent estimator $t_n$ tends to zero is of great importance in large sample theory of point estimation. The main tools available at present for finding the rate are Bernstein-Chernoff-Bahadur's theorem and Sanov's theorem. In this paper, we give two new techniques for finding the rate of convergence of certain consistent estimators. By using these techniques, we have obtained an upper bound for the rate of convergence of consistent estimators based on sample quantities and proved that the sample median is an asymptotically efficient estimator in Bahadur's sense if and only if the underlying distribution is double-exponential. Furthermore, we have proved that the Bahadur asymptotic relative efficiency of sample mean and sample median coincides with the classical Pitman asymptotic relative efficiency.

https://projecteuclid.org/download/pdf_1/euclid.aos/1176343013

James C. Fu

Papers

Large Sample Point Estimation: A Large Deviation Theory Approach

On a Theorem of Bahadur on the Rate of Convergence of Point Estimators

The Rate of Convergence of Consistent Point Estimators