We propose an independence criterion based on the eigen-spectrum of covariance operators in reproducing kernel Hilbert spaces (RKHSs), consisting of an empirical estimate of the Hilbert-Schmidt norm of the cross-covariance operator (we term this a Hilbert-Schmidt Independence Criterion, or HSIC). This approach has several advantages, compared with previous kernel-based independence criteria. First, the empirical estimate is simpler than any other kernel dependence test, and requires no user-defined regularisation. Second, there is a clearly defined population quantity which the empirical estimate approaches in the large sample limit, with exponential convergence guaranteed between the two: this ensures that independence tests based on HSIC do not suffer from slow learning rates. Finally, we show in the context of independent component analysis (ICA) that the performance of HSIC is competitive with that of previously published kernel-based criteria, and of other recently published ICA methods.

Measuring statistical dependence with Hilbert-Schmidt norms

With the advance of modern technology, more and more data are being recorded continuously during a time interval or intermittently at several discrete time points. These are both examples of functional data, which has become a commonly encountered type of data. Functional data analysis (FDA) encompasses the statistical methodology for such data. Broadly interpreted, FDA deals with the analysis and theory of data that are in the form of functions. This paper provides an overview of FDA, starting with simple statistical notions such as mean and covariance functions, then covering some core techniques, the most popular of which is functional principal component analysis (FPCA). FPCA is an important dimension reduction tool, and in sparse data situations it can be used to impute functional data that are sparsely observed. Other dimension reduction approaches are also discussed. In addition, we review another core technique, functional linear regression, as well as clustering and classification of functional d...

/pdf/functional-data-analysis-2yrey8q6kx.pdf

Functional Data Analysis

While the additive model is a popular nonparametric regression method, many of its theoretical properties are not well understood, especially when the backfitting algorithm is used for computation of the estimators. This article explores those properties when the additive model is fitted by local polynomial regression. Sufficient conditions guaranteeing the asymptotic existence of unique estimators for the bivariate additive model are given. Asymptotic approximations to the bias and the variance of a homoscedastic bivariate additive model with local polynomial terms of odd and even degree are computed. This model is shown to have the same rate of convergence as that of univariate local polynomial regression.

Fitting a Bivariate Additive Model by Local Polynomial Regression

Ten Lectures on Wavelets Volume 921

Recent advances in functional data analysis and high-dimensional statistics

Nonparametric estimation of mean and covariance functions is important in functional data analysis. We investigate the performance of local linear smoothers for both mean and covariance functions with a general weighing scheme, which includes two commonly used schemes, equal weight per observation (OBS), and equal weight per subject (SUBJ), as two special cases. We provide a comprehensive analysis of their asymptotic properties on a unified platform for all types of sampling plan, be it dense, sparse or neither. Three types of asymptotic properties are investigated in this paper: asymptotic normality, $L^{2}$ convergence and uniform convergence. The asymptotic theories are unified on two aspects: (1) the weighing scheme is very general; (2) the magnitude of the number $N_{i}$ of measurements for the $i$th subject relative to the sample size $n$ can vary freely. Based on the relative order of $N_{i}$ to $n$, functional data are partitioned into three types: non-dense, dense and ultra-dense functional data for the OBS and SUBJ schemes. These two weighing schemes are compared both theoretically and numerically. We also propose a new class of weighing schemes in terms of a mixture of the OBS and SUBJ weights, of which theoretical and numerical performances are examined and compared.

/pdf/from-sparse-to-dense-functional-data-and-beyond-21gz2zs221.pdf

From sparse to dense functional data and beyond

The additive model is an effective dimension-reduction approach that also provides flexibility in modeling the relation between a response variable and key covariates. The literature is largely developed to scalar response and vector covariates. In this article, more complex data are of interest, where both the response and the covariates are functions. We propose a functional additive model together with a new backfitting algorithm to estimate the unknown regression functions, whose components are time-dependent additive functions of the covariates. Such functional data may not be completely observed since measurements may only be collected intermittently at discrete time points. We develop a unified platform and an efficient approach that can cover both dense and sparse functional data and the needed theory for statistical inference. We also establish the oracle properties of the proposed estimators of the component functions.

/pdf/time-varying-additive-models-for-longitudinal-data-215ccyvxx4.pdf

Time-Varying Additive Models for Longitudinal Data

Ongoing variability in neural signaling is an intrinsic property of the brain. Often this variability is considered to be noise and ignored. However, an alternative view is that this variability is fundamental to perception and cognition and may be particularly important in decision-making. Here, we show that a momentary measure of occipital alpha-band power (8—13 Hz) predicts choices about where human participants will focus spatial attention on a trial-by-trial basis. This finding provides evidence for a mechanistic account of decision-making by demonstrating that ongoing neural activity biases voluntary decisions about where to attend within a given moment.

/pdf/spontaneous-neural-fluctuations-predict-decisions-to-attend-4zupy74tr5.pdf

Spontaneous neural fluctuations predict decisions to attend

Functional data analysis (FDA) is concerned with inherently infinite-dimensional data objects and therefore can be viewed as part of the methodology for big data. The size of functional data may vary from terabytes as encountered in functional magnetic resonance imaging (fMRI) and other applications in brain imaging to just a few kilobytes in longitudinal data with small or modest sample sizes. In this contribution, we highlight some applications of FDA methodology through various data illustrations. We briefly review some basic computational tools that can be used to accelerate implementations of FDA methodology. The analyses presented in this paper illustrate the principal analysis by conditional expectation (PACE) package for FDA, where our applications include both relatively simple and more complex functional data from the biomedical sciences. The data we discuss range from functional data that result from daily movement profile tracking and that are modeled as repeatedly observed functions per subject, to medfly longitudinal behavior profiles, where the goal is to predict remaining lifetime of individual flies. We also discuss the quantification of connectivity of fMRI signals that is of interest in brain imaging and the prediction of continuous traits from high-dimensional SNPs in genomics. The methods of FDA that we demonstrate for these analyses include functional principal component analysis, functional regression and correlation, the modeling of dependent functional data, and the stringing of high-dimensional data into functional data and can be implemented with the PACE package.

/pdf/quantifying-infinite-dimensional-data-functional-data-e7teqxxqwe.pdf

Quantifying Infinite-Dimensional Data: Functional Data Analysis in Action

Both varying-coe cient and additive models have been studied extensively in the literature as extensions to linear models. They have also been extended to functional response data. However, existing extensions are still not su ciently flexible to reflect the functional feature of the responses. In this paper, we extend both varying-coe cient and additive models to a much more flexible “varying-coe cient additive model” and propose a simple algorithm to estimate the nonparametric additive and varying-coe cient components of this model. We establish the L 2 rate of convergence for each component function and demonstrate the applicability of the new model through tra c data.

/pdf/varying-coefficient-additive-models-for-functional-data-43u9vbkwsv.pdf

Xiaoke Zhang

Papers

From sparse to dense functional data and beyond

Time-Varying Additive Models for Longitudinal Data

Spontaneous neural fluctuations predict decisions to attend

Quantifying Infinite-Dimensional Data: Functional Data Analysis in Action

Varying-coefficient additive models for functional data