scispace - formally typeset
Search or ask a question

Showing papers on "Gaussian process published in 2012"


Posted Content
TL;DR: In this paper, a learning algorithm's generalization performance is modeled as a sample from a Gaussian process and the tractable posterior distribution induced by the GP leads to efficient use of the information gathered by previous experiments, enabling optimal choices about what parameters to try next.
Abstract: Machine learning algorithms frequently require careful tuning of model hyperparameters, regularization terms, and optimization parameters. Unfortunately, this tuning is often a "black art" that requires expert experience, unwritten rules of thumb, or sometimes brute-force search. Much more appealing is the idea of developing automatic approaches which can optimize the performance of a given learning algorithm to the task at hand. In this work, we consider the automatic tuning problem within the framework of Bayesian optimization, in which a learning algorithm's generalization performance is modeled as a sample from a Gaussian process (GP). The tractable posterior distribution induced by the GP leads to efficient use of the information gathered by previous experiments, enabling optimal choices about what parameters to try next. Here we show how the effects of the Gaussian process prior and the associated inference procedure can have a large impact on the success or failure of Bayesian optimization. We show that thoughtful choices can lead to results that exceed expert-level performance in tuning machine learning algorithms. We also describe new algorithms that take into account the variable cost (duration) of learning experiments and that can leverage the presence of multiple cores for parallel experimentation. We show that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization on a diverse set of contemporary algorithms including latent Dirichlet allocation, structured SVMs and convolutional neural networks.

1,110 citations


Journal ArticleDOI
TL;DR: This work analyzes an intuitive Gaussian process upper confidence bound algorithm, and bound its cumulative regret in terms of maximal in- formation gain, establishing a novel connection between GP optimization and experimental design and obtaining explicit sublinear regret bounds for many commonly used covariance functions.
Abstract: Many applications require optimizing an unknown, noisy function that is expensive to evaluate. We formalize this task as a multiarmed bandit problem, where the payoff function is either sampled from a Gaussian process (GP) or has low norm in a reproducing kernel Hilbert space. We resolve the important open problem of deriving regret bounds for this setting, which imply novel convergence rates for GP optimization. We analyze an intuitive Gaussian process upper confidence bound (GP-UCB) algorithm, and bound its cumulative regret in terms of maximal in- formation gain, establishing a novel connection between GP optimization and experimental design. Moreover, by bounding the latter in terms of operator spectra, we obtain explicit sublinear regret bounds for many commonly used covariance functions. In some important cases, our bounds have surprisingly weak dependence on the dimensionality. In our experiments on real sensor data, GP-UCB compares favorably with other heuristical GP optimization approaches.

851 citations


Journal ArticleDOI
TL;DR: A dual mathematical interpretation of the proposed framework with a structured sparse estimation is described, which shows that the resulting piecewise linear estimate stabilizes the estimation when compared with traditional sparse inverse problem techniques.
Abstract: A general framework for solving image inverse problems with piecewise linear estimations is introduced in this paper. The approach is based on Gaussian mixture models, which are estimated via a maximum a posteriori expectation-maximization algorithm. A dual mathematical interpretation of the proposed framework with a structured sparse estimation is described, which shows that the resulting piecewise linear estimate stabilizes the estimation when compared with traditional sparse inverse problem techniques. We demonstrate that, in a number of image inverse problems, including interpolation, zooming, and deblurring of narrow kernels, the same simple and computationally efficient algorithm yields results in the same ballpark as that of the state of the art.

505 citations


Journal ArticleDOI
TL;DR: A classical regularization approach, focused on finite impulse response (FIR) models, is formulated, and it is found that regularization is necessary to cope with the high variance problem.

458 citations


Journal ArticleDOI
TL;DR: In this paper, a non-parametric approach to reconstructing the history of the expansion rate and dark energy using Gaussian Processes, which is a fully Bayesian approach for smoothing data, is presented.
Abstract: An important issue in cosmology is reconstructing the effective dark energy equation of state directly from observations. With few physically motivated models, future dark energy studies cannot only be based on constraining a dark energy parameter space, as the errors found depend strongly on the parametrisation considered. We present a new non-parametric approach to reconstructing the history of the expansion rate and dark energy using Gaussian Processes, which is a fully Bayesian approach for smoothing data. We present a pedagogical introduction to Gaussian Processes, and discuss how it can be used to robustly differentiate data in a suitable way. Using this method we show that the Dark Energy Survey - Supernova Survey (DES) can accurately recover a slowly evolving equation of state to σw = ±0.05 (95% CL) at z = 0 and ±0.25 at z = 0.7, with a minimum error of ±0.025 at the sweet-spot at z ~ 0.16, provided the other parameters of the model are known. Errors on the expansion history are an order of magnitude smaller, yet make no assumptions about dark energy whatsoever. A code for calculating functions and their first three derivatives using Gaussian processes has been developed and is available for download.

359 citations


Journal ArticleDOI
TL;DR: In this paper, it was shown that the normalized risk of the LASSO converges to a limit, and an explicit expression for this limit was derived for random instances, based on the analysis of AMP.
Abstract: We consider the problem of learning a coefficient vector xο ∈ RN from noisy linear observation y = Axo + ∈ Rn. In many contexts (ranging from model selection to image processing), it is desirable to construct a sparse estimator x. In this case, a popular approach consists in solving an l1-penalized least-squares problem known as the LASSO or basis pursuit denoising. For sequences of matrices A of increasing dimensions, with independent Gaussian entries, we prove that the normalized risk of the LASSO converges to a limit, and we obtain an explicit expression for this limit. Our result is the first rigorous derivation of an explicit formula for the asymptotic mean square error of the LASSO for random instances. The proof technique is based on the analysis of AMP, a recently developed efficient algorithm, that is inspired from graphical model ideas. Simulations on real data matrices suggest that our results can be relevant in a broad array of practical applications.

334 citations


Journal ArticleDOI
TL;DR: SUR (stepwise uncertainty reduction) strategies are derived from a Bayesian formulation of the problem of estimating a probability of failure of a function f using a Gaussian process model of f and aim at performing evaluations of f as efficiently as possible to infer the value of the probabilities of failure.
Abstract: This paper deals with the problem of estimating the volume of the excursion set of a function f:? d ?? above a given threshold, under a probability measure on ? d that is assumed to be known. In the industrial world, this corresponds to the problem of estimating a probability of failure of a system. When only an expensive-to-simulate model of the system is available, the budget for simulations is usually severely limited and therefore classical Monte Carlo methods ought to be avoided. One of the main contributions of this article is to derive SUR (stepwise uncertainty reduction) strategies from a Bayesian formulation of the problem of estimating a probability of failure. These sequential strategies use a Gaussian process model of f and aim at performing evaluations of f as efficiently as possible to infer the value of the probability of failure. We compare these strategies to other strategies also based on a Gaussian process model for estimating a probability of failure.

330 citations


Journal ArticleDOI
TL;DR: In this paper, the authors proposed a nonparametric Gaussian Process (GP) method to infer transit parameters in the presence of systematic noise using Gaussian processes, a technique widely used in the machine learning community for Bayesian regression and classification problems.
Abstract: Transmission spectroscopy, which consists of measuring the wavelength-dependent absorption of starlight by a planet’s atmosphere during a transit, is a powerful probe of atmospheric composition. However, the expected signal is typically orders of magnitude smaller than instrumental systematics and the results are crucially dependent on the treatment of the latter. In this paper, we propose a new method to infer transit parameters in the presence of systematic noise using Gaussian processes, a technique widely used in the machine learning community for Bayesian regression and classification problems. Our method makes use of auxiliary information about the state of the instrument, but does so in a non-parametric manner, without imposing a specific dependence of the systematics on the instrumental parameters, and naturally allows for the correlated nature of the noise. We give an example application of the method to archival NICMOS transmission spectroscopy of the hot Jupiter HD 189733, which goes some way towards reconciling the controversy surrounding this data set in the literature. Finally, we provide an appendix giving a general introduction to Gaussian processes for regression, in order to encourage their application to a wider range of problems.

328 citations


Book ChapterDOI
01 Jan 2012
TL;DR: This chapter describes the Gaussian random process, the subject of this chapter, which has many important properties that have been inherited from those of the N-dimensional Gaussian PDF, which was discussed in Section 14.3.
Abstract: There are several types of random processes that have found wide application because of their realistic physical modeling yet relative mathematical simplicity. In this and the next two chapters we describe these important random processes. They are the Gaussian random process, the subject of this chapter; the Poisson random process, described in Chapter 21; and the Markov chain, described in Chapter 22. Concentrating now on the Gaussian random process, we will see that it has many important properties. These properties have been inherited from those of the N-dimensional Gaussian PDF, which was discussed in Section 14.3.

314 citations


Journal ArticleDOI
TL;DR: This paper presents an alternative approach to pseudo measurement modeling in the context of distribution system state estimation (DSSE), where pseudo measurements are generated from a few real measurements using artificial neural networks in conjunction with typical load profiles.
Abstract: This paper presents an alternative approach to pseudo measurement modeling in the context of distribution system state estimation (DSSE). In the proposed approach, pseudo measurements are generated from a few real measurements using artificial neural networks (ANNs) in conjunction with typical load profiles. The error associated with the generated pseudo measurements is made suitable for use in the weighted least squares (WLS) state estimation by decomposition into several components through the Gaussian mixture model (GMM). The effect of ANN-based pseudo measurement modeling on the quality of state estimation is demonstrated on a 95-bus section of the U.K. generic distribution system (UKGDS) model.

299 citations


Journal ArticleDOI
TL;DR: It is observed that hierarchical kriging provides a more reasonable mean-squared-error estimation than traditional cokriging and can be applied to the efficient aerodynamic analysis and shape optimization of aircraft or anywhere where computer codes of varying fidelity are in use.
Abstract: The efficiency of building a surrogate model for the output of a computer code can be dramatically improved via variable-fidelity surrogate modeling techniques. In this article, a hierarchical kriging model is proposed and used for variable-fidelity surrogate modeling problems. Here, hierarchical kriging refers to a surrogate model of a highfidelity function that uses a kriging model of a sampled lower-fidelity function as a model trend. As a consequence, the variation in the lower-fidelity data is mapped to the high-fidelity data, and a more accurate surrogate model for the high-fidelity function is obtained. A self-contained derivation of the hierarchical kriging model is presented. The proposed method is demonstrated with an analytical example and used for modeling the aerodynamic data of an RAE 2822 airfoil and an industrial transport aircraft configuration. The numerical examples show that it is efficient, accurate, and robust. It is also observed that hierarchical kriging provides a more reasonable mean-squared-error estimation than traditional cokriging. It can be applied to the efficient aerodynamic analysis and shape optimization of aircraft or any other research areas where computer codes of varying fidelity are in use.

Book
16 Oct 2012
TL;DR: In this paper, the authors studied the convergence rate for Levy's modulus of continuity and Hinchin's law of the iterated logarithm for U-statistics of order two.
Abstract: I Measures on General Spaces and Inequalities- Stochastic inequalities and perfect independence- Prokhorov-LeCam-Varadarajan's compactness criteria for vector measures on metric spaces- On measures in locally convex spaces- II Gaussian Processes- Karhunen-Loeve expansions for weighted Wiener processes and Brownian bridges via Bessel functions- Extension du theoreme de Cameron-Martin aux translations aleatoires II Integrabilite des densites- III Limit Theorems- Rates of convergence for Levy's modulus of continuity and Hinchin's law of the iterated logarithm- On the limit set in the law of the iterated logarithm for U-statistics of order two- Perturbation approach applied to the asymptotic study of random operators- A uniform functional law of the logarithm for a local Gaussian process- Strong limit theorems for mixing random variables with values in Hilbert space and their applications- IV Local Times- Local time-space calculus and extensions of Ito's formula- Local times on curves and surfaces- V Large, Small Deviations- Large deviations of empirical processes- Small deviation estimates for some additive processes- VI Density Estimation- Convergence in distribution of self-normalized sup-norms of kernel density estimators- Estimates of the rate of approximation in the CLT for L1-norm of density estimators- VII Statistics via Empirical Process Theory- Statistical nearly universal Glivenko-Cantelli classes- Smoothed empirical processes and the bootstrap- A note on the asymptotic distribution of Berk-Jones type statistics under the null hypothesis- A note on the smoothed bootstrap

Journal ArticleDOI
TL;DR: The technique can handle noisy data, potentially from multiple sources, and fuse it into a robust common probabilistic representation of the robot’s surroundings, and provides inferences with associated variances into occluded regions and between sensor beams, even with relatively few observations.
Abstract: We introduce a new statistical modelling technique for building occupancy maps. The problem of mapping is addressed as a classification task where the robot's environment is classified into regions of occupancy and free space. This is obtained by employing a modified Gaussian process as a non-parametric Bayesian learning technique to exploit the fact that real-world environments inherently possess structure. This structure introduces dependencies between points on the map which are not accounted for by many common mapping techniques such as occupancy grids. Our approach is an 'anytime' algorithm that is capable of generating accurate representations of large environments at arbitrary resolutions to suit many applications. It also provides inferences with associated variances into occluded regions and between sensor beams, even with relatively few observations. Crucially, the technique can handle noisy data, potentially from multiple sources, and fuse it into a robust common probabilistic representation of the robot's surroundings. We demonstrate the benefits of our approach on simulated datasets with known ground truth and in outdoor urban environments.

Proceedings ArticleDOI
16 Jun 2012
TL;DR: This work proposes a patch based approach, where it is shown that the light field patches with the same disparity value lie on a low-dimensional subspace and that the dimensionality of such subspaces varies quadratically with the disparity value.
Abstract: With the recent availability of commercial light field cameras, we can foresee a future in which light field signals will be as common place as images. Hence, there is an imminent need to address the problem of light field processing. We provide a common framework for addressing many of the light field processing tasks, such as denoising, angular and spatial superresolution, etc. (in essence, all processing tasks whose observation models are linear). We propose a patch based approach, where we model the light field patches using a Gaussian mixture model (GMM). We use the ”disparity pattern” of the light field data to design the patch prior. We show that the light field patches with the same disparity value (i.e., at the same depth from the focal plane) lie on a low-dimensional subspace and that the dimensionality of such subspaces varies quadratically with the disparity value. We then model the patches as Gaussian random variables conditioned on its disparity value, thus, effectively leading to a GMM model. During inference, we first find the disparity value of a patch by a fast subspace projection technique and then reconstruct it using the LMMSE algorithm. With this prior and inference algorithm, we show that we can perform many different processing tasks under a common framework.


Journal ArticleDOI
TL;DR: In this article, the time evolution of the cosmic expansion is mapped using kinematic variables and a geometric probe of cosmology using supernova distance data from the Union2.1 compilation.
Abstract: Gaussian processes provide a method for extracting cosmological information from observations without assuming a cosmological model. We carry out cosmography---mapping the time evolution of the cosmic expansion---in a model-independent manner using kinematic variables and a geometric probe of cosmology. Using the state of the art supernova distance data from the Union2.1 compilation, we constrain, without any assumptions about dark energy parametrization or matter density, the Hubble parameter and deceleration parameter as a function of redshift. Extraction of these relations is tested successfully against models with features on various coherence scales, subject to certain statistical cautions.

Journal ArticleDOI
TL;DR: In this paper, the authors describe a prototype approach to flexible modelling for maxima observed at sites in a spatial domain, based on fitting of max-stable processes derived from underlying Gaussian random fields.
Abstract: We describe a prototype approach to flexible modelling for maxima observed at sites in a spatial domain, based on fitting of max-stable processes derived from underlying Gaussian random fields. The models we propose have generalized extreme-value marginal distributions throughout the spatial domain, consistent with statistical theory for maxima in simpler cases, and can incorporate both geostatistical correlation functions and random set components. Parameter estimation and fitting are performed through composite likelihood inference applied to observations from pairs of sites, with occurrence times of maxima taken into account if desired, and competing models are compared using appropriate information criteria. Diagnostics for lack of model fit are based on maxima from groups of sites. The approach is illustrated using annual maximum temperatures in Switzerland, with risk analysis proposed using simulations from the fitted max-stable model. Drawbacks and possible developments of the approach are discussed.

Journal ArticleDOI
TL;DR: In this article, a non-parametric approach to reconstructing the history of the expansion rate and dark energy using Gaussian Processes is presented, which is a fully Bayesian approach for smoothing data.
Abstract: An important issue in cosmology is reconstructing the effective dark energy equation of state directly from observations. With few physically motivated models, future dark energy studies cannot only be based on constraining a dark energy parameter space, as the errors found depend strongly on the parameterisation considered. We present a new non-parametric approach to reconstructing the history of the expansion rate and dark energy using Gaussian Processes, which is a fully Bayesian approach for smoothing data. We present a pedagogical introduction to Gaussian Processes, and discuss how it can be used to robustly differentiate data in a suitable way. Using this method we show that the Dark Energy Survey - Supernova Survey (DES) can accurately recover a slowly evolving equation of state to sigma_w = +-0.04 (95% CL) at z=0 and +-0.2 at z=0.7, with a minimum error of +-0.015 at the sweet-spot at z~0.14, provided the other parameters of the model are known. Errors on the expansion history are an order of magnitude smaller, yet make no assumptions about dark energy whatsoever. A code for calculating functions and their first three derivatives using Gaussian processes has been developed and is available for download at this http URL .

Proceedings ArticleDOI
21 Mar 2012
TL;DR: This work proposes an empirical-Bayesian technique that simultaneously learns the signal distribution while MMSE-recovering the signal-according to the learned distribution-using AMP, and model the non-zero distribution as a Gaussian mixture and learn its parameters through expectation maximization, using AMP to implement the expectation step.
Abstract: When recovering a sparse signal from noisy compressive linear measurements, the distribution of the signal's non-zero coefficients can have a profound affect on recovery mean-squared error (MSE). If this distribution was apriori known, one could use efficient approximate message passing (AMP) techniques for nearly minimum MSE (MMSE) recovery. In practice, though, the distribution is unknown, motivating the use of robust algorithms like Lasso—which is nearly minimax optimal—at the cost of significantly larger MSE for non-least-favorable distributions. As an alternative, we propose an empirical-Bayesian technique that simultaneously learns the signal distribution while MMSE-recovering the signal—according to the learned distribution—using AMP. In particular, we model the non-zero distribution as a Gaussian mixture, and learn its parameters through expectation maximization, using AMP to implement the expectation step. Numerical experiments confirm the state-of-the-art performance of our approach on a range of signal classes.

Journal ArticleDOI
TL;DR: Comment on the errors in the formulation of Theorem 1 and give a correct formulation of theorem.
Abstract: This paper presents a Gaussian-mixture (GM) implementation of the probability hypothesis density (PHD) filter for tracking extended targets. The exact filter requires processing of all possible measurement set partitions, which is generally infeasible to implement. A method is proposed for limiting the number of considered partitions and possible alternatives are discussed. The implementation is used on simulated data and in experiments with real laser data, and the advantage of the filter is illustrated. Suitable remedies are given to handle spatially close targets and target occlusion.

Journal ArticleDOI
TL;DR: This paper proposes to use the multivariate Gaussian source prior to achieve JBSS of sources that are linearly dependent across datasets, and introduces both Newton and quasi-Newton optimization algorithms for the general IVA framework.
Abstract: In this paper, we consider the joint blind source separation (JBSS) problem and introduce a number of algorithms to solve the JBSS problem using the independent vector analysis (IVA) framework. Source separation of multiple datasets simultaneously is possible when the sources within each and every dataset are independent of one another and each source is dependent on at most one source within each of the other datasets. In addition to source separation, the IVA framework solves an essential problem of JBSS, namely the identification of the dependent sources across the datasets. We propose to use the multivariate Gaussian source prior to achieve JBSS of sources that are linearly dependent across datasets. Analysis within the paper yields the local stability conditions, nonidentifiability conditions, and induced Cramer-Rao lower bound on the achievable interference to source ratio for IVA with multivariate Gaussian source priors. Additionally, by exploiting a novel nonorthogonal decoupling of the IVA cost function we introduce both Newton and quasi-Newton optimization algorithms for the general IVA framework.

Journal ArticleDOI
TL;DR: In this paper, a distributed method for computing, at each sensor, an approximation of the joint likelihood function (JLF) by means of consensus algorithms is proposed, which is applicable if the local likelihood functions of the various sensors (viewed as conditional probability density functions of local measurements) belong to the exponential family of distributions.
Abstract: We consider distributed state estimation in a wireless sensor network without a fusion center. Each sensor performs a global estimation task-based on the past and current measurements of all sensors-using only local processing and local communications with its neighbors. In this estimation task, the joint (all-sensors) likelihood function (JLF) plays a central role as it epitomizes the measurements of all sensors. We propose a distributed method for computing, at each sensor, an approximation of the JLF by means of consensus algorithms. This “likelihood consensus” method is applicable if the local likelihood functions of the various sensors (viewed as conditional probability density functions of the local measurements) belong to the exponential family of distributions. We then use the likelihood consensus method to implement a distributed particle filter and a distributed Gaussian particle filter. Each sensor runs a local particle filter, or a local Gaussian particle filter, that computes a global state estimate. The weight update in each local (Gaussian) particle filter employs the JLF, which is obtained through the likelihood consensus scheme. For the distributed Gaussian particle filter, the number of particles can be significantly reduced by means of an additional consensus scheme. Simulation results are presented to assess the performance of the proposed distributed particle filters for a multiple target tracking problem.

Journal ArticleDOI
TL;DR: Two modeling approaches to combine monitoring data at pointlevel with numerical model output at grid cell level are proposed, yielding improved prediction of ambient exposure at point level, and the predictive gain is greater at hold-out sites that are far from monitoring sites.
Abstract: Summary. We provide methods that can be used to obtain more accurate environmental exposure assessment. In particular, we propose two modeling approaches to combine monitoring data at point level with numerical model output at grid cell level, yielding improved prediction of ambient exposure at point level. Extending our earlier downscaler model (Berrocal, V. J., Gelfand, A. E., and Holland, D. M. (2010b). A spatio-temporal downscaler for outputs from numerical models. Journal of Agricultural, Biological and Environmental Statistics 15, 176–197), these new models are intended to address two potential concerns with the model output. One recognizes that there may be useful information in the outputs for grid cells that are neighbors of the one in which the location lies. The second acknowledges potential spatial misalignment between a station and its putatively associated grid cell. The first model is a Gaussian Markov random field smoothed downscaler that relates monitoring station data and computer model output via the introduction of a latent Gaussian Markov random field linked to both sources of data. The second model is a smoothed downscaler with spatially varying random weights defined through a latent Gaussian process and an exponential kernel function, that yields, at each site, a new variable on which the monitoring station data is regressed with a spatial linear model. We applied both methods to daily ozone concentration data for the Eastern US during the summer months of June, July and August 2001, obtaining, respectively, a 5% and a 15% predictive gain in overall predictive mean square error over our earlier downscaler model (Berrocal et al., 2010b). Perhaps more importantly, the predictive gain is greater at hold-out sites that are far from monitoring sites.

Proceedings ArticleDOI
01 Dec 2012
TL;DR: A new matrix completion algorithm— Kernelized Probabilistic Matrix Factorization (KPMF), which effectively incorporates external side information into the matrix factorization process, and can be easily used with other types of side information as well.
Abstract: We propose a new matrix completion algorithm— Kernelized Probabilistic Matrix Factorization (KPMF), which effectively incorporates external side information into the matrix factorization process. Unlike Probabilistic Matrix Factorization (PMF) [14], which assumes an independent latent vector for each row (and each column) with Gaussian priors, KMPF works with latent vectors spanning all rows (and columns) with Gaussian Process (GP) priors. Hence, KPMF explicitly captures the underlying (nonlinear) covariance structures across rows and columns. This crucial difference greatly boosts the performance of KPMF when appropriate side information, e.g., users’ social network in recommender systems, is incorporated. Furthermore, GP priors allow the KPMF model to fill in a row that is entirely missing in the original matrix based on the side information alone, which is not feasible for standard PMF formulation. In our paper, we mainly work on the matrix completion problem with a graph among the rows and/or columns as side information, but the proposed framework can be easily used with other types of side information as well. Finally, we demonstrate the efficacy of KPMF through two different applications: 1) recommender systems and 2) image restoration.

Journal ArticleDOI
TL;DR: An efficient, Bayesian Uncertainty Quantification framework using a novel treed Gaussian process model is developed and numerically demonstrate the effectiveness of the suggested framework in identifying discontinuities, local features and unimportant dimensions in the solution of stochastic differential equations.

Journal ArticleDOI
TL;DR: In this article, a daily stochastic spatio-temporal precipitation generator that yields spatially consistent gridded quantitative precipitation realizations is described, which relies on a latent Gaussian process to drive precipitation occurrence and a probability integral transformed Gaussian processes for intensity.
Abstract: [1] A daily stochastic spatiotemporal precipitation generator that yields spatially consistent gridded quantitative precipitation realizations is described. The methodology relies on a latent Gaussian process to drive precipitation occurrence and a probability integral transformed Gaussian process for intensity. At individual locations, the model reduces to a Markov chain for precipitation occurrence and a gamma distribution for precipitation intensity, allowing statistical parameters to be included in a generalized linear model framework. Statistical parameters are modeled as spatial Gaussian processes, which allows for interpolation to locations where there are no direct observations via kriging. One advantage of such a model for the statistical parameters is that stochastic generator parameters are immediately available at any location, with the ability to adapt to spatially varying precipitation characteristics. A second advantage is that parameter uncertainty, generally unavailable with deterministic interpolators, can be immediately quantified at all locations. The methodology is illustrated on two data sets, the first in Iowa and the second over the Pampas region of Argentina. In both examples, the method is able to capture the local and domain aggregated precipitation behavior fairly well at a wide range of time scales, including daily, monthly, and annually.

Journal ArticleDOI
TL;DR: It is shown that, in functional data classification problems, perfect asymptotic classification is often possible, making use of the intrinsic very high dimensional nature of functional data, which points to a marked contrast between classification for functional data and its counterpart in conventional multivariate analysis.
Abstract: Summary. We show that, in functional data classification problems, perfect asymptotic classification is often possible, making use of the intrinsic very high dimensional nature of functional data. This performance is often achieved by linear methods, which are optimal in important cases. These results point to a marked contrast between classification for functional data and its counterpart in conventional multivariate analysis, where the dimension is kept fixed as the sample size diverges. In the latter setting, linear methods can sometimes be quite inefficient, and there are no prospects for asymptotically perfect classification, except in pathological cases where, for example, a variance vanishes. By way of contrast, in finite samples of functional data, good performance can be achieved by truncated versions of linear methods. Truncation can be implemented by partial least squares or projection onto a finite number of principal components, using, in both cases, cross-validation to determine the truncation point. We establish consistency of the cross-validation procedure.

Journal ArticleDOI
TL;DR: The developed cokriging method is validated against an analytical problem and applied to construct global approximation models of the aerodynamic coefficients as well as the drag polar of an RAE 2822 airfoil.
Abstract: Surrogate modeling plays an increasingly important role in different areas of aerospace engineering, such as erodynamic shape optimization, aerodynamic data production, structural design, and multidisciplinary design optimization of aircraft or spacecraft. Cokriging provides an attractive alternative approach to conventional kriging to improve the efficiency of building a surrogate model. It was initially proposed and applied in the geostatistics community for the enhanced prediction of less intensively sampled primary variables of interest with the assistance of intensively sampled auxiliary variables. As the underlying theory of cokriging is that of two-variable or multivariable kriging, it can be regarded as a general extension of (one-variable) kriging to a model that is assisted by auxiliary variables or secondary information. In an attempt to apply cokriging to the surrogate modeling problems associated with deterministic computer experiments, this article is motivated by the development of an alternative cokriging method to address the challenge related to the construction of the covariance matrix of cokriging [7]. Earlier work done by other authors related to this study can be found in the statistical community. For example, Kennedy and O’Hagan (KOH) proposed an autoregressive model to calculate the covariances and crosscovariances in the covariance matrix and developed a Bayesian approach to predict the output from an expensive high-fidelity simulation code with the assistance of lower-fidelity simulation codes. This Bayesian approach is identical to a form of cokriging suitable for computer experiments. Later, Qian andWu proposed a similar method, in which a random function (Gaussian process model) was used to replace the constant multiplicative factor of KOH’s method to account for the nonlinear scale change. KOH’s method was applied to multifidelity analysis and design optimization in the context of aerospace engineering by Forrester et al. and Kuya et al. More recently, Zimmerman and Han proposed a cokriging method with simplified cross-correlation estimation. In this article, we propose an alternative approach for the construction of the cokriging covariance matrix and develop a more practical cokriging method in the context of surrogate-based analysis and optimization. The developed cokriging method is validated against an analytical problem and applied to construct global approximation models of the aerodynamic coefficients as well as the drag polar of an RAE 2822 airfoil.

Book
13 Jan 2012
TL;DR: The theory of Gaussian processes occupies one of the leading places in modern Probability as discussed by the authors, which is why Gaussian vectors and Gaussian distributions in infinite-dimensional spaces come into play.
Abstract: Theory of random processes needs a kind of normal distribution. This is why Gaussian vectors and Gaussian distributions in infinite-dimensional spaces come into play. By simplicity, importance and wealth of results, theory of Gaussian processes occupies one of the leading places in modern Probability.

Journal ArticleDOI
TL;DR: This paper analyzes the performance of Kalman filtering for discrete-time linear Gaussian systems, where packets containing observations are dropped according to a Markov process modeling a Gilbert-Elliot channel and shows that the trace of the Kalman estimation error covariance under intermittent observations follows a power decay law.
Abstract: In this paper, we analyze the performance of Kalman filtering for discrete-time linear Gaussian systems, where packets containing observations are dropped according to a Markov process modeling a Gilbert-Elliot channel. To address the challenges incurred by the loss of packets, we give a new definition of non-degeneracy, which is essentially stronger than the classical definition of observability, but much weaker than one-step observability, which is usually used in the study of Kalman filtering with intermittent observations. We show that the trace of the Kalman estimation error covariance under intermittent observations follows a power decay law. Moreover, we are able to compute the exact decay rate for non-degenerate systems. Finally, we derive the critical value for non-degenerate systems based on the decay rate, improving upon the state of the art.