Probability and Measure

We propose an independence criterion based on the eigen-spectrum of covariance operators in reproducing kernel Hilbert spaces (RKHSs), consisting of an empirical estimate of the Hilbert-Schmidt norm of the cross-covariance operator (we term this a Hilbert-Schmidt Independence Criterion, or HSIC). This approach has several advantages, compared with previous kernel-based independence criteria. First, the empirical estimate is simpler than any other kernel dependence test, and requires no user-defined regularisation. Second, there is a clearly defined population quantity which the empirical estimate approaches in the large sample limit, with exponential convergence guaranteed between the two: this ensures that independence tests based on HSIC do not suffer from slow learning rates. Finally, we show in the context of independent component analysis (ICA) that the performance of HSIC is competitive with that of previously published kernel-based criteria, and of other recently published ICA methods.

Measuring statistical dependence with Hilbert-Schmidt norms

“Multivalued Analysis” is the theory of set-valued maps (called multifonctions) and has important applications in many different areas. Multivalued analysis is a remarkable mixture of many different parts of mathematics such as point-set topology, measure theory and nonlinear functional analysis. It is also closely related to “Nonsmooth Analysis” (Chapter 5) and in fact one of the main motivations behind the development of the theory, was in order to provide necessary analytical tools for the study of problems in nonsmooth analysis. It is not a coincidence that the development of the two fields coincide chronologically and follow parallel paths. Today multivalued analysis is a mature mathematical field with its own methods, techniques and applications that range from social and economic sciences to biological sciences and engineering. There is no doubt that a modern treatise on “Nonlinear Functional Analysis” can not afford the luxury of ignoring multivalued analysis. The omission of the theory of multifunctions will drastically limit the possible applications.

SET-Valued Analysis

Review: Divergence measures for statistical data processing-An annotated bibliography

We propose a new method of effective dimension reduction for a multiindex model which is based on iterative improvement of the family of average derivative estimates. The procedure is computationally straightforward and does not require any prior information about the structure of the underlying model. We show that in the case when the effective dimension m of the index space does not exceed 3, this space can be estimated with the rate n −1/2 under rather mild assumptions on the model.

Structure Adaptative Approach for Dimension Reduction

We consider a two-component mixture model with one known component. We develop methods for estimating the mixing proportion and the unknown distribution nonparametrically, given i.i.d.~data from the mixture model, using ideas from shape restricted function estimation. We establish the consistency of our estimators. We find the rate of convergence and asymptotic limit of the estimator for the mixing proportion. Completely automated distribution-free honest finite sample lower confidence bounds are developed for the mixing proportion. Connection to the problem of multiple testing is discussed. The identifiability of the model, and the estimation of the density of the unknown distribution are also addressed. We compare the proposed estimators, which are easily implementable, with some of the existing procedures through simulation studies and analyse two data sets, one arising from an application in astronomy and the other from a microarray experiment.

Estimation of a Two-component Mixture Model with Applications to Multiple Testing

We consider a two-component mixture model with one known component. We develop methods for estimating the mixing proportion and the other unknown distribution nonparametrically, given i.i.d. data from the mixture model. We use ideas from shape restricted function estimation and develop “tuning parameter free” estimators that are easily implementable and have good finite sample performance. We establish the consistency of our procedures. Distribution-free finite sample lower confidence bounds are developed for the mixing proportion. The identifiability of the model, and the estimation of the density of the unknown mixing distribution are also addressed. We discuss the connection with the problem of multiple testing and compare our procedure with some of the existing methods in that area through simulation studies. We also analyse two data sets, one arising from an application in astronomy and the other from a microarray experiment.

/pdf/estimation-of-a-two-component-mixture-model-with-2hkin85ysj.pdf

Tidal debris structures formed from disrupted satellites contain important clues about the assembly histories of galaxies. To date, studies of these structures have been hampered by reliance on by-eye identification and morphological classification which leaves their interpretation significantly uncertain. In this work we present a new machine-vision technique based on the Subspace-Constrained Mean Shift (SCMS) algorithm which can perform these tasks automatically. SCMS finds the location of the high-density `ridges' that define substructure morphology. After identification, the coefficients of an orthogonal series density estimator are used to classify points on the ridges as part of a continuum between shell-like or stream-like debris, from which a global morphological classification can be determined. We dub this procedure Subspace Constrained Unsupervised Detection of Structure (SCUDS). By applying this tool to controlled N--body simulations of minor mergers we demonstrate that the extracted classifications correspond to the well--understood underlying physics of phase mixing. The application of SCUDS to resolved stellar population data from near-future surveys will inform our understanding of the buildup of galaxies stellar halos.

A machine-vision method for automatic classification of stellar halo substructure

We consider estimation and inference in a single index regression model with an unknown but smooth link function. In contrast to the standard approach of using kernels or regression splines, we use smoothing splines to estimate the smooth link function. We develop a method to compute the penalized least squares estimators (PLSEs) of the parametric and the nonparametric components given independent and identically distributed (i.i.d.)~data. We prove the consistency and find the rates of convergence of the estimators. We establish asymptotic normality under under mild assumption and prove asymptotic efficiency of the parametric component under homoscedastic errors. A finite sample simulation corroborates our asymptotic theory. We also analyze a car mileage data set and a Ozone concentration data set. The identifiability and existence of the PLSEs are also investigated.

Rohit K. Patra

Papers

Estimation of a Two-component Mixture Model with Applications to Multiple Testing

Estimation of a Two-component Mixture Model with Applications to Multiple Testing

A machine-vision method for automatic classification of stellar halo substructure

Efficient Estimation in Single Index Models through Smoothing splines

A consistent bootstrap procedure for the maximum score estimator