Showing papers in "Biometrika in 2011"
TL;DR: The square-root lasso as mentioned in this paper is a modification of the lasso, which achieves near-oracle performance, attaining the convergence rate in the prediction norm, and matching the performance of the Lasso with known standard deviation.
Abstract: We propose a pivotal method for estimating high-dimensional sparse linear regression models, where the overall number of regressors $p$ is large, possibly much larger than $n$, but only $s$ regressors are significant. The method is a modification of the lasso, called the square-root lasso. The method is pivotal in that it neither relies on the knowledge of the standard deviation $\sigma$ or nor does it need to pre-estimate $\sigma$. Moreover, the method does not rely on normality or sub-Gaussianity of noise. It achieves near-oracle performance, attaining the convergence rate $\sigma \{(s/n)\log p\}^{1/2}$ in the prediction norm, and thus matching the performance of the lasso with known $\sigma$. These performance results are valid for both Gaussian and non-Gaussian errors, under some mild moment restrictions. We formulate the square-root lasso as a solution to a convex conic programming problem, which allows us to implement the estimator using efficient algorithmic methods, such as interior-point and first-order methods.
510 citations
TL;DR: An estimator forGaussian graphical models appropriate for data from several graphical models that share the same variables and some of the dependence structure is developed, aiming to preserve the common structure, while allowing for differences between the categories.
Abstract: Gaussian graphical models explore dependence relationships between random variables, through the estimation of the corresponding inverse covariance matrices. In this paper we develop an estimator for such models appropriate for data from several graphical models that share the same variables and some of the dependence structure. In this setting, estimating a single graphical model would mask the underlying heterogeneity, while estimating separate models for each category does not take advantage of the common structure. We propose a method that jointly estimates the graphical models corresponding to the different categories present in the data, aiming to preserve the common structure, while allowing for differences between the categories. This is achieved through a hierarchical penalty that targets the removal of common zeros in the inverse covariance matrices across categories. We establish the asymptotic consistency and sparsity of the proposed estimator in the high-dimensional case, and illustrate its performance on a number of simulated networks. An application to learning semantic connections between terms from webpages collected from computer science departments is included.
465 citations
TL;DR: This work proposes a multiplicative gamma process shrinkage prior on the factor loadings which allows introduction of infinitely many factors, with the loadings increasingly shrunk towards zero as the column index increases, and develops an efficient Gibbs sampler that scales well as data dimensionality increases.
Abstract: We focus on sparse modelling of high-dimensional covariance matrices using Bayesian latent factor models. We propose a multiplicative gamma process shrinkage prior on the factor loadings which allows introduction of infinitely many factors, with the loadings increasingly shrunk towards zero as the column index increases. We use our prior on a parameter-expanded loading matrix to avoid the order dependence typical in factor analysis models and develop an efficient Gibbs sampler that scales well as data dimensionality increases. The gain in efficiency is achieved by the joint conjugacy property of the proposed prior, which allows block updating of the loadings matrix. We propose an adaptive Gibbs sampler for automatically truncating the infinite loading matrix through selection of the number of important factors. Theoretical results are provided on the support of the prior and truncation approximation bounds. A fast algorithm is proposed to produce approximate Bayes estimates. Latent factor regression methods are developed for prediction and variable selection in applications with high-dimensional correlated predictors. Operating characteristics are assessed through simulation studies, and the approach is applied to predict survival times from gene expression data.
439 citations
TL;DR: The proposed penalized maximum likelihood problem is not convex, so the method can be used to solve a previously studied special case in which a desired sparsity pattern is prespecified, and it uses a majorize-minimize approach in which it iteratively solve convex approximations to the original nonconvex problem.
Abstract: We suggest a method for estimating a covariance matrix on the basis of a sample of vectors drawn from a multivariate normal distribution. In particular, we penalize the likelihood with a lasso penalty on the entries of the covariance matrix. This penalty plays two important roles: it reduces the effective number of parameters, which is important even when the dimension of the vectors is smaller than the sample size since the number of parameters grows quadratically in the number of variables, and it produces an estimate which is sparse. In contrast to sparse inverse covariance estimation, our method’s close relative, the sparsity attained here is in the covariance matrix itself rather than in the inverse matrix. Zeros in the covariance matrix correspond to marginal independencies; thus, our method performs model selection while providing a positive definite estimate of the covariance. The proposed penalized maximum likelihood problem is not convex, so we use a majorize-minimize approach in which we iteratively solve convex approximations to the original nonconvex problem. We discuss tuning parameter selection and demonstrate on a flow-cytometry dataset how our method produces an interpretable graphical display of the relationship between variables. We perform simulations that suggest that simple elementwise thresholding of the empirical covariance matrix is competitive with our method for identifying the sparsity structure. Additionally, we show how our method can be used to solve a previously studied special case in which a desired sparsity pattern is prespecified.
307 citations
TL;DR: It is shown that the fraction of misclassified network nodes converges in probability to zero under maximum likelihood fitting when the number of classes is allowed to grow as the root of the network size and the average network degree grows at least poly-logarithmically in this size.
Abstract: We present asymptotic and finite-sample results on the use of stochastic blockmodels for the analysis of network data. We show that the fraction of misclassified network nodes converges in probability to zero under maximum likelihood fitting when the number of classes is allowed to grow as the root of the network size and the average network degree grows at least poly-logarithmically in this size. We also establish finite-sample confidence bounds on maximum-likelihood blockmodel parameter estimates from data comprising independent Bernoulli random variates; these results hold uniformly over class assignment. We provide simulations verifying the conditions sufficient for our results, and conclude by fitting a logit parameterization of a stochastic blockmodel with covariates to a network data example comprising self-reported school friendships, resulting in block estimates that reveal residual structure.
274 citations
TL;DR: This work presents two particle algorithms to compute the score vector and observed information matrix recursively in nonlinear non-Gaussian state space models and shows how both methods can be used to perform batch and recursive parameter estimation.
Abstract: Particle methods are popular computational tools for Bayesian inference in nonlinear non-Gaussian state space models. For this class of models, we present two particle algorithms to compute the score vector and observed information matrix recursively. The first algorithm is implemented with computational complexity O(N) and the second with complexity O(N 2 ), where N is the number of particles. Although cheaper, the performance of the O(N) method degrades quickly, as it relies on the approximation of a sequence of probability distributions whose dimension increases linearly with time. In particular, even under strong mixing assumptions, the variance of the estimates computed with the O(N) method increases at least quadratically in time. The more expensive O(N 2 ) method relies on a nonstandard particle implementation and does not suffer from this rapid degradation. It is shown how both methods can be used to perform batch and recursive parameter estimation.
265 citations
TL;DR: In this paper, the control of unmeasured confounders in parametric and nonparametric models and the computational problem of obtaining bias-free effect estimates in such models are discussed.
Abstract: SUMMARY This paper highlights several areas where graphical techniques can be harnessed to address the problem of measurement errors in causal inference. In particular, it discusses the control of unmeasured confounders in parametric and nonparametric models and the computational problem of obtaining bias-free effect estimates in such models. We derive new conditions under which causal effects can be restored by observing proxy variables of unmeasured confounders with/without external studies.
199 citations
TL;DR: In this paper, the dimension reduction of high-dimensional time series based on common factors is discussed. But the authors consider the case where the dimension of the time series p is as large as or even larger than the sample size n. The estimation of the factor loading matrix and the factor process itself is carried out via an eigenanalysis of a p £ p nonnegative de¯nite matrix.
Abstract: This paper deals with the dimension reduction of high-dimensional time series based on common factors. In particular we allow the dimension of time series p to be as large as, or even larger than, the sample size n. The estimation of the factor loading matrix and the factor process itself is carried out via an eigenanalysis of a p £ p non-negative de¯nite matrix. We show that when all the factors are strong in the sense that the norm of each column in the factor loading matrix is of the order p1=2, the estimator of the factor loading matrix is weakly consistent in L2-norm with the convergence rate independent of p. This result exhibits clearly that the `curse' is canceled out by the `blessing' of dimensionality. We also establish the asymptotic properties of the estimation when factors are not strong. The proposed method together with their asymptotic properties are further illustrated in a simulation study. An application to an implied volatility data set, together with a trading strategy derived from the ¯tted factor model, is also reported.
162 citations
TL;DR: This note proposes a modification of the wild bootstrap that admits a broader class of weight distributions that is asymptotically valid for quantile regression estimators and is shown to account for general forms of heteroscedasticity in a regression model with fixed design points.
Abstract: The existing theory of the wild bootstrap has focused on linear estimators. In this note, we broaden its validity by providing a class of weight distributions that is asymptotically valid for quantile regression estimators. As most weight distributions in the literature lead to biased variance estimates for nonlinear estimators of linear regression, we propose a modification of the wild bootstrap that admits a broader class of weight distributions for quantile regression. A simulation study on median regression is carried out to compare various bootstrap methods. With a simple finite-sample correction, the wild bootstrap is shown to account for general forms of heteroscedasticity in a regression model with fixed design points.
154 citations
TL;DR: Observational studies in which the effect of a non-randomized treatment on an outcome of interest is estimated are common in domains such as labour economics and epidemiology as mentioned in this paper.
Abstract: Observational studies in which the effect of a nonrandomized treatment on an outcome of interest is estimated are common in domains such as labour economics and epidemiology. Such studies often rel ...
137 citations
TL;DR: In this article, the maximum composite likelihood estimators of the covariance matrix from p = 2t op = 3 sites in R 2 were obtained by means of a Monte Carlo method.
Abstract: SUMMARY WederiveaclosedformexpressionforthelikelihoodfunctionofaGaussianmax-stableprocessindexed by R d at p d + 1 sites, d 1. We demonstrate the gain in efficiency in the maximum composite likelihood estimators of the covariance matrix from p = 2t op =3 sites in R 2 by means of a Monte Carlo
TL;DR: It is shown that correlation may increase the bias and variance of the estimators substantially with respect to the independent case, and that in some cases, such as an exchangeable correlation structure, the estimator fails to be consistent as the number of tests becomes large.
Abstract: The objective of this paper is to quantify the effect of correlation in false discovery rate analysis. Specifically, we derive approximations for the mean, variance, distribution and quantiles of the standard false discovery rate estimator for arbitrarily correlated data. This is achieved using a negative binomial model for the number of false discoveries, where the parameters are found empirically from the data. We show that correlation may increase the bias and variance of the estimator substantially with respect to the independent case, and that in some cases, such as an exchangeable correlation structure, the estimator fails to be consistent as the number of tests becomes large.
TL;DR: In this paper, a parametric fractional imputation (FPI) method is proposed to generate imputed values from the conditional distribution of the missing data given the observed data, where the fractional weights are computed from the current value of the parameter estimates.
Abstract: Under a parametric model for missing data, the EM algorithm is a popular tool for flnding the maximum likelihood estimates (MLE) of the parameters of the model. Imputation, when carefully done, can be used to facilitate the parameter estimation by applying the complete-sample estimators to the imputed dataset. The basic idea is to generate the imputed values from the conditional distribution of the missing data given the observed data. Multiple imputation is a Bayesian approach to generate the imputed values from the conditional distribution. In this article, parametric fractional imputation is proposed as a parametric approach for generating imputed values. Using fractional weights, the E-step of the EM algorithm can be approximated by the weighted mean of the imputed data likelihood where the fractional weights are computed from the current value of the parameter estimates. Some computational e‐ciency can be achieved using the idea of importance sampling in the Monte Carlo approximation of the conditional expectation. The resulting estimator of the specifled parameters will be identical to the MLE under missing data if the fractional weights are adjusted using a calibration step. The proposed imputation method provides e‐cient parameter estimates for the model parameters specifled and also provides reasonable estimates for parameters that are not part of the imputation model, for example domain means. Thus, the proposed imputation method is a useful tool for general-purpose data analysis. Variance estimation is covered and results from a limited simulation study are presented.
TL;DR: A Bayesian approach is proposed, which models the locations using a log Gaussian Cox process, while modelling the outcomes conditionally on the locations as Gaussian with a Gaussian process spatial random effect and adjustment for the location intensity process.
Abstract: We consider geostatistical models that allow the locations at which data are collected to be informative about the outcomes. A Bayesian approach is proposed, which models the locations using a log Gaussian Cox process, while modelling the outcomes conditionally on the locations as Gaussian with a Gaussian process spatial random effect and adjustment for the location intensity process. We prove posterior propriety under an improper prior on the parameter controlling the degree of informative sampling, demonstrating that the data are informative. In addition, we show that the density of the locations and mean function of the outcome process can be estimated consistently under mild assumptions. The methods show significant evidence of informative sampling when applied to ozone data over Eastern U.S.A.
TL;DR: A class of dependent processes in which density shape is regressed on one or more predictors through conditional tail-free probabilities by using transformed Gaussian processes is proposed, which is flexible and easy to fit using standard algorithms for generalized linear models.
Abstract: We propose a class of dependent processes in which density shape is regressed on one or more predictors through conditional tail-free probabilities by using transformed Gaussian processes. A particular linear version of the process is developed in detail. The resulting process is flexible and easy to fit using standard algorithms for generalized linear models. The method is applied to growth curve analysis, evolving univariate random effects distributions in generalized linear mixed models, and median survival modelling with censored data and covariate-dependent errors.
TL;DR: The proposed criteria are applied to two cancer datasets to select models when the cluster-specific inference is of interest, and show that the performance of the bootstrap and the analytic criteria are comparable.
Abstract: We study model selection for clustered data, when the focus is on cluster specific inference. Such data are often modelled using random effects, and conditional Akaike information was proposed in Vaida & Blanchard (2005) and used to derive an information criterion under linear mixed models. Here we extend the approach to generalized linear and proportional hazards mixed models. Outside the normal linear mixed models, exact calculations are not available and we resort to asymptotic approximations. In the presence of nuisance parameters, a profile conditional Akaike information is proposed. Bootstrap methods are considered for their potential advantage in finite samples. Simulations show that the performance of the bootstrap and the analytic criteria are comparable, with bootstrap demonstrating some advantages for larger cluster sizes. The proposed criteria are applied to two cancer datasets to select models when the cluster-specific inference is of interest.
TL;DR: In this paper, the authors show that the distribution of such a test statistic can be approximated by a ratio of quadratic forms in normal variables, for which algorithms are readily available.
Abstract: Testing a low-dimensional null hypothesis against a high-dimensional alternative in a generalized linear model may lead to a test statistic that is a quadratic form in the residuals under the null model. Using asymptotic arguments, we show that the distribution of such a test statistic can be approximated by a ratio of quadratic forms in normal variables, for which algorithms are readily available. For generalized linear models, the asymptotic distribution shows good control of type I error for moderate to small samples, even when the number of covariates in the model far exceeds the sample size.
TL;DR: An algorithm for estimating the parameters in a finite mixture of completely unspecified multivariate components in at least three dimensions under the assumption of conditionally independent coordinate dimensions is introduced and it is proved that this algorithm possesses a desirable descent property just as any em algorithm does.
Abstract: SUMMARY We introduce an algorithm for estimating the parameters in a finite mixture of completely unspecified multivariate components in at least three dimensions under the assumption of conditionally independent coordinate dimensions. We prove that this algorithm, based on a majorization-minimization idea, possesses a desirable descent property just as any EM algorithm does. We discuss the similarities between our algorithm and a related one, the so-called nonlinearly smoothed EM algorithm for the non-mixture setting. We also demonstrate via simulation studies that the new algorithm gives very similar results to another algorithm that has been shown empirically to be effective but that does not satisfy any descent property. We provide code for implementing the new algorithm in a publicly available R package.
TL;DR: This work examines approaches for estimating and controlling the false discovery rate, and provides examples from biological applications.
Abstract: The false discovery rate is a criterion for controlling Type I error in simultaneous testing of multiple hypotheses. For scanning statistics, due to local dependence, clusters of neighbouring hypotheses are likely to be rejected together. In such situations, it is more intuitive and informative to group neighbouring rejections together and count them as a single discovery, with the false discovery rate defined as the proportion of clusters that are falsely declared among all declared clusters. Assuming that the number of false discoveries, under this broader definition of a discovery, is approximately Poisson and independent of the number of true discoveries, we examine approaches for estimating and controlling the false discovery rate, and provide examples from biological applications. Copyright 2011, Oxford University Press.
TL;DR: A nonparametric estimator is proposed that incorporates the information about the length-biased sampling scheme and retains the simplicity of the truncation product-limit estimator with a closed-form expression, and has a small efficiency loss compared with thenonparametric maximum likelihood estimator.
Abstract: This paper considers survival data arising from length-biased sampling, where the survival times are left truncated by uniformly distributed random truncation times. We propose a nonparametric estimator that incorporates the information about the length-biased sampling scheme. The new estimator retains the simplicity of the truncation product-limit estimator with a closed-form expression, and has a small efficiency loss compared with the nonparametric maximum likelihood estimator, which requires an iterative algorithm. Moreover, the asymptotic variance of the proposed estimator has a closed form, and a variance estimator is easily obtained by plug-in methods. Numerical simulation studies with practical sample sizes are conducted to compare the performance of the proposed method with its competitors. A data analysis of the Canadian Study of Health and Aging is conducted to illustrate the methods and theory.
TL;DR: In this paper, the authors proposed a method that combines the efficient semiparametric estimator with nonparametric covariance estimation, and is robust against misspecification of covariance models.
Abstract: SUMMARY For longitudinal data, when the within-subject covariance is misspecified, the semiparametric regression estimator may be inefficient. We propose a method that combines the efficient semiparametric estimator with nonparametric covariance estimation, and is robust against misspecification of covariance models. We show that kernel covariance estimation provides uniformly consistent estimators for the within-subject covariance matrices, and the semiparametric profile estimator with substituted nonparametric covariance is still semiparametrically efficient. The finite sample performance of the proposed estimator is illustrated by simulation. In an application to CD4 count data from an AIDS clinical trial, we extend the proposed method to a functional analysis of the covariance model.
TL;DR: The main contribution is a general method for constructing randomized trial designs that allow changes to the population enrolled based on interim data using a prespecified decision rule, for which the asymptotic, familywise Type I error rate is strongly controlled at a specified level α.
Abstract: It is a challenge to evaluate experimental treatments where it is suspected that the treatment effect may only be strong for certain subpopulations, such as those having a high initial severity of disease, or those having a particular gene variant. Standard randomized controlled trials can have low power in such situations. They also are not optimized to distinguish which subpopulations benefit from a treatment. With the goal of overcoming these limitations, we consider randomized trial designs in which the criteria for patient enrollment may be changed, in a preplanned manner, based on interim analyses. Since such designs allow data-dependent changes to the population enrolled, care must be taken to ensure strong control of the familywise Type I error rate. Our main contribution is a general method for constructing randomized trial designs that allow changes to the population enrolled based on interim data using a prespecified decision rule, for which the asymptotic, familywise Type I error rate is strongly controlled at a specified level α. As a demonstration of our method, we prove new, sharp results for a simple, two-stage enrichment design. We then compare this design to fixed designs, focusing on each design’s ability to determine the overall and subpopulation-specific treatment effects.
TL;DR: In this paper, a construction principle for the spectral density of a multivariate extreme value distribution is presented. But it is based on the pairwise beta model introduced in the literature recently and may be used to obtain new parametric models from lower dimensional spectral densities.
Abstract: We present a construction principle for the spectral density of a multivariate extreme value distribution. It generalizes the pairwise beta model introduced in the literature recently and may be used to obtain new parametric models from lower dimensional spectral densities. We illustrate the flexibility of this new class of models and apply it to a wind speed dataset.
TL;DR: In this paper, a new family of covariate-adaptive randomized designs that represent higher order approximation to balance treatments, both globally and also across covariates, is proposed. But the performance of the proposed designs is compared with those of other procedures suggested in the literature.
Abstract: The present paper deals with sequential designs intended to balance the allocations of two competing treatments in the presence of prognostic factors. After giving a theoretical framework on the optimality of balanced designs that can arise when covariates are taken into account, we propose a new family of covariate-adaptive randomized designs that represents higher order approximation to balance treatments, both globally and also across covariates. We derive the theoretical properties of the suggested designs in terms of loss of precision and predictability. The performance of this proposal is illustrated through a simulation study and compared with those of other procedures suggested in the literature. Copyright 2011, Oxford University Press.
TL;DR: This article proposed a novel quantile regression approach for longitudinal data analysis which naturally incorporates auxiliary information from the conditional mean model to account for within-subject correlations, and demonstrated the efficiency gain is quantified theoretically and demonstrated empirically via simulation studies and the analysis of a real dataset.
Abstract: We propose a novel quantile regression approach for longitudinal data analysis which naturally incorporates auxiliary information from the conditional mean model to account for within-subject correlations. The efficiency gain is quantified theoretically and demonstrated empirically via simulation studies and the analysis of a real dataset.
TL;DR: In this article, the authors proposed an analternative Bayesian approach based on empirical likelihood which does not require the assumption of either a parametric likelihoodor linearity of estimators.
Abstract: Small areaestimation has become a topic of growingimportance inrecent years due to increasing demand for small area statistics fromboth the public and private sectors. The current methodologies inthis area with possibly a handful of exceptions, are all parametric,be it Bayesian or frequentist. Even the so-called
onparamet-ric" procedures based on empirical best linear unbiased predictors(EBLUP) assume linearity of the estimators of small area means.Resampling methods like jacknife and bootstrap are used mostlyfor mean squared error estimation. In this paper we propose analternative Bayesian approach based on empirical likelihood whichdoes not require the assumption of either a parametric likelihoodor linearity of estimators. The proposed method can also handleboth discrete and continuous data in a uni ed manner. Empiricallikelihood for both area and unit level models are introduced. Theimproved performance of our estimator is illustrated through twoexamples involving both discrete and continuous data.Key words and phrases. Area level; Exponentially tilted; t-prior;Unit level.
TL;DR: A general framework of Bayesian influence analysis for assessing various perturbation schemes to the data, the prior and the sampling distribution for a class of statistical models is developed.
Abstract: In this paper we develop a general framework of Bayesian influence analysis for assessing various perturbation schemes to the data, the prior and the sampling distribution for a class of statistical models. We introduce a perturbation model to characterize these various perturbation schemes. We develop a geometric framework, called the Bayesian perturbation manifold, and use its associated geometric quantities including the metric tensor and geodesic to characterize the intrinsic structure of the perturbation model. We develop intrinsic influence measures and local influence measures based on the Bayesian perturbation manifold to quantify the effect of various perturbations to statistical models. Theoretical and numerical examples are examined to highlight the broad spectrum of applications of this local influence method in a formal Bayesian analysis.
TL;DR: This paper examines the effect of ignoring response-dependent, informative, cluster sizes on standard analytical methods such as mixed-effects models and conditional likelihood methods using analytic calculations, simulation studies and an example from a study of periodontal disease.
Abstract: In standard regression analyses of clustered data, one typically assumes that the expected value of the response is independent of cluster size. However, this is often false. For example, in studies of surgical interventions, investigators have frequently found surgery volume and outcomes to be related to the skill level of the surgeons. This paper examines the effect of ignoring response-dependent, informative, cluster sizes on standard analytical methods such as mixed-effects models and conditional likelihood methods using analytic calculations, simulation studies and an example from a study of periodontal disease. We consider the case in which cluster sizes and responses share random effects which we assume to be independent of the covariates. Our focus is on maximum likelihood methods that ignore informative cluster sizes, and we show that they exhibit little bias in estimating covariate effects that are uncorrelated with the random effects associated with cluster sizes. However, estimation of covariate effects that are associated with the random effects can be biased. In particular, for models with random intercepts only, ignoring informative cluster sizes can yield biased estimators of the intercept but little bias in estimation of all covariate effects.
TL;DR: In this paper, a bias-reducing penalized maximum likelihood estimator for a multinomial logistic regression model using the equivalent Poisson log-linear model is presented.
Abstract: For the parameters of a multinomial logistic regression, it is shown how to obtain the bias-reducing penalized maximum likelihood estimator by using the equivalent Poisson log-linear model. The calculation needed is not simply an application of the Jeffreys prior penalty to the Poisson model. The development allows a simple and computationally efficient implementation of the reduced-bias estimator, using standard software for generalized linear models.
TL;DR: In this paper, a central limit theorem for the integrated squared error is derived, and a hypothesis-testing procedure is proposed to assess the mean pattern of lifetime-maximum wind speeds of global tropical cyclones from 1981 to 2006.
Abstract: The paper considers testing whether the mean trend of a nonstationary time series is of certain parametric forms. A central limit theorem for the integrated squared error is derived, and a hypothesis-testing procedure is proposed. The method is illustrated in a simulation study, and is applied to assess the mean pattern of lifetime-maximum wind speeds of global tropical cyclones from 1981 to 2006. We also revisit the trend pattern in the central England temperature series. Copyright 2011, Oxford University Press.