Showing papers on "Parametric statistics published in 2014"

Face Alignment by Explicit Shape Regression

Abstract: We present a very efficient, highly accurate, "Explicit Shape Regression" approach for face alignment. Unlike previous regression-based approaches, we directly learn a vectorial regression function to infer the whole facial shape (a set of facial landmarks) from the image and explicitly minimize the alignment errors over the training data. The inherent shape constraint is naturally encoded into the regressor in a cascaded learning framework and applied from coarse to fine during the test, without using a fixed parametric shape model as in most previous methods. To make the regression more effective and efficient, we design a two-level boosted regression, shape indexed features and a correlation-based feature selection method. This combination enables us to learn accurate models from large training data in a short time (20 min for 2,000 training images), and run regression extremely fast in test (15 ms for a 87 landmarks shape). Experiments on challenging data show that our approach significantly outperforms the state-of-the-art in terms of both accuracy and efficiency.

qpOASES: a parametric active-set algorithm for quadratic programming

Abstract: Many practical applications lead to optimization problems that can either be stated as quadratic programming (QP) problems or require the solution of QP problems on a lower algorithmic level. One relatively recent approach to solve QP problems are parametric active-set methods that are based on tracing the solution along a linear homotopy between a QP problem with known solution and the QP problem to be solved. This approach seems to make them particularly suited for applications where a-priori information can be used to speed-up the QP solution or where high solution accuracy is required. In this paper we describe the open-source C++ software package qpOASES, which implements a parametric active-set method in a reliable and efficient way. Numerical tests show that qpOASES can outperform other popular academic and commercial QP solvers on small- to medium-scale convex test examples of the Maros-Meszaros QP collection. Moreover, various interfaces to third-party software packages make it easy to use, even on embedded computer hardware. Finally, we describe how qpOASES can be used to compute critical points of nonconvex QP problems.

A Kenward-Roger approximation and parametric bootstrap methods for tests in linear mixed models: The R Package pbkrtest

Abstract: When testing for reduction of the mean value structure in linear mixed models, it is common to use an asymptotic ?2 test. Such tests can, however, be very poor for small and moderate sample sizes. The pbkrtest package implements two alternatives to such approximate ?2 tests: The package implements (1) a Kenward-Roger approximation for performing F tests for reduction of the mean structure and (2) parametric bootstrap methods for achieving the same goal. The implementation is focused on linear mixed models with independent residual errors. In addition to describing the methods and aspects of their implementation, the paper also contains several examples and a comparison of the various methods.

Genome-Wide Regression and Prediction with the BGLR Statistical Package

Abstract: Many modern genomic data analyses require implementing regressions where the number of parameters (p, e.g., the number of marker effects) exceeds sample size (n). Implementing these large-p-with-small-n regressions poses several statistical and computational challenges, some of which can be confronted using Bayesian methods. This approach allows integrating various parametric and nonparametric shrinkage and variable selection procedures in a unified and consistent manner. The BGLR R-package implements a large collection of Bayesian regression models, including parametric variable selection and shrinkage methods and semiparametric procedures (Bayesian reproducing kernel Hilbert spaces regressions, RKHS). The software was originally developed for genomic applications; however, the methods implemented are useful for many nongenomic applications as well. The response can be continuous (censored or not) or categorical (either binary or ordinal). The algorithm is based on a Gibbs sampler with scalar updates and the implementation takes advantage of efficient compiled C and Fortran routines. In this article we describe the methods implemented in BGLR, present examples of the use of the package, and discuss practical issues emerging in real-data analysis.

Robust Point Matching via Vector Field Consensus

Abstract: In this paper, we propose an efficient algorithm, called vector field consensus, for establishing robust point correspondences between two sets of points. Our algorithm starts by creating a set of putative correspondences which can contain a very large number of false correspondences, or outliers, in addition to a limited number of true correspondences (inliers). Next, we solve for correspondence by interpolating a vector field between the two point sets, which involves estimating a consensus of inlier points whose matching follows a nonparametric geometrical constraint. We formulate this a maximum a posteriori (MAP) estimation of a Bayesian model with hidden/latent variables indicating whether matches in the putative set are outliers or inliers. We impose nonparametric geometrical constraints on the correspondence, as a prior distribution, using Tikhonov regularizers in a reproducing kernel Hilbert space. MAP estimation is performed by the EM algorithm which by also estimating the variance of the prior model (initialized to a large value) is able to obtain good estimates very quickly (e.g., avoiding many of the local minima inherent in this formulation). We illustrate this method on data sets in 2D and 3D and demonstrate that it is robust to a very large number of outliers (even up to 90%). We also show that in the special case where there is an underlying parametric geometrical model (e.g., the epipolar line constraint) that we obtain better results than standard alternatives like RANSAC if a large number of outliers are present. This suggests a two-stage strategy, where we use our nonparametric model to reduce the size of the putative set and then apply a parametric variant of our approach to estimate the geometric parameters. Our algorithm is computationally efficient and we provide code for others to use it. In addition, our approach is general and can be applied to other problems, such as learning with a badly corrupted training data set.

Estimation and Accuracy after Model Selection.

Abstract: Classical statistical theory ignores model selection in assessing estimation accuracy. Here we consider bootstrap methods for computing standard errors and confidence intervals that take model selection into account. The methodology involves bagging, also known as bootstrap smoothing, to tame the erratic discontinuities of selection-based estimators. A useful new formula for the accuracy of bagging then provides standard errors for the smoothed estimators. Two examples, nonparametric and parametric, are carried through in detail: a regression model where the choice of degree (linear, quadratic, cubic, …) is determined by the Cp criterion and a Lasso-based estimation problem.

Locally Optimized Product Quantization for Approximate Nearest Neighbor Search

Abstract: We present a simple vector quantizer that combines low distortion with fast search and apply it to approximate nearest neighbor (ANN) search in high dimensional spaces. Leveraging the very same data structure that is used to provide non-exhaustive search, i.e., inverted lists or a multi-index, the idea is to locally optimize an individual product quantizer (PQ) per cell and use it to encode residuals. Local optimization is over rotation and space decomposition, interestingly, we apply a parametric solution that assumes a normal distribution and is extremely fast to train. With a reasonable space and time overhead that is constant in the data size, we set a new state-of-the-art on several public datasets, including a billion-scale one.

On the Spectral Efficiency of Full-Duplex Small Cell Wireless Systems

Abstract: We investigate the spectral efficiency of full-duplex small cell wireless systems, in which a full-duplex capable base station (BS) is designed to send/receive data to/from multiple half-duplex users on the same system resources. The major hurdle for designing such systems is due to the self-interference at the BS and co-channel interference among users. Hence, we consider a joint beamformer design to maximize the spectral efficiency subject to certain power constraints. The design problem is first formulated as a rank-constrained optimization problem, and the rank relaxation method is then applied. However, the relaxed problem is still nonconvex, and thus, optimal solutions are hard to find. Herein, we propose two provably convergent algorithms to obtain suboptimal solutions. Based on the concept of the Frank-Wolfe algorithm, we approximate the design problem by a determinant maximization program in each iteration of the first algorithm. The second method is built upon the sequential parametric convex approximation method, which allows us to transform the relaxed problem into a semidefinite program in each iteration. Extensive numerical experiments under small cell setups illustrate that the full-duplex system with the proposed algorithms can achieve a large gain over the half-duplex system.

Deep mixture density networks for acoustic modeling in statistical parametric speech synthesis

Abstract: Statistical parametric speech synthesis (SPSS) using deep neural networks (DNNs) has shown its potential to produce naturally-sounding synthesized speech. However, there are limitations in the current implementation of DNN-based acoustic modeling for speech synthesis, such as the unimodal nature of its objective function and its lack of ability to predict variances. To address these limitations, this paper investigates the use of a mixture density output layer. It can estimate full probability density functions over real-valued output features conditioned on the corresponding input features. Experimental results in objective and subjective evaluations show that the use of the mixture density output layer improves the prediction accuracy of acoustic features and the naturalness of the synthesized speech.

On the Spectral Efficiency of Full-Duplex Small Cell Wireless Systems

Abstract: We investigate the spectral efficiency of full-duplex small cell wireless systems, in which a full-duplex capable base station (BS) is designed to send/receive data to/from multiple halfduplex users on the same system resources. The major hurdle for designing such systems is due to the self-interference at the BS and co-channel interference among users. Hence, we consider a joint beamformer design to maximize the spectral efficiency subject to certain power constraints. The design problem is first formulated as a rank-constrained optimization one, and the rank relaxation method is then applied. However the relaxed problem is still nonconvex, and thus optimal solutions are hard to find. Herein, we propose two provably convergent algorithms to obtain suboptimal solutions. Based on the concept of the difference of convex functions programming, we approximate the design problem by a determinant maximization program in each iteration of the first algorithm. The second method is built upon the sequential parametric convex approximation method, which allows us to transform the relaxed problem into a semidefinite program in each iteration. Extensive numerical experiments under small cell setups illustrate that the full-duplex system with the proposed algorithms can achieve a large gain over the half-duplex one.

Stochastic nonlinear model predictive control with probabilistic constraints

Abstract: Stochastic uncertainties are ubiquitous in complex dynamical systems and can lead to undesired variability of system outputs and, therefore, a notable degradation of closed- loop performance. This paper investigates model predictive control of nonlinear dynamical systems subject to probabilistic parametric uncertainties. A nonlinear model predictive control framework is presented for control of the probability dis- tribution of system states while ensuring the satisfaction of constraints with some desired probability levels. To obtain a computationally tractable formulation for real control applica- tions, polynomial chaos expansions are utilized to propagate the probabilistic parametric uncertainties through the system model. The paper considers individual probabilistic constraints, which are converted explicitly into convex second-order cone constraints for a general class of probability distributions. An algorithm is presented for receding horizon implementation of the finite-horizon stochastic optimal control problem. The capability of the stochastic model predictive control approach in terms of shaping the probability distribution of system states and fulfilling state constraints in a stochastic setting is demon- strated for optimal control of polymorphic transformation in batch crystallization.

A method for assessing phylogenetic least squares models for shape and other high-dimensional multivariate data

Abstract: Studies of evolutionary correlations commonly use phylogenetic regression (i.e., independent contrasts and phylogenetic generalized least squares) to assess trait covariation in a phylogenetic context. However, while this approach is appropriate for evaluating trends in one or a few traits, it is incapable of assessing patterns in highly multivariate data, as the large number of variables relative to sample size prohibits parametric test statistics from being computed. This poses serious limitations for comparative biologists, who must either simplify how they quantify phenotypic traits, or alter the biological hypotheses they wish to examine. In this article, I propose a new statistical procedure for performing ANOVA and regression models in a phylogenetic context that can accommodate high-dimensional datasets. The approach is derived from the statistical equivalency between parametric methods using covariance matrices and methods based on distance matrices. Using simulations under Brownian motion, I show that the method displays appropriate Type I error rates and statistical power, whereas standard parametric procedures have decreasing power as data dimensionality increases. As such, the new procedure provides a useful means of assessing trait covariation across a set of taxa related by a phylogeny, enabling macroevolutionary biologists to test hypotheses of adaptation, and phenotypic change in high-dimensional datasets.

Wind Turbine Power Curve Modeling Using Advanced Parametric and Nonparametric Methods

Abstract: Wind turbine power curve modeling is an important tool in turbine performance monitoring and power forecasting. There are several statistical techniques to fit the empirical power curve of a wind turbine, which can be classified into parametric and nonparametric methods. In this paper, we study four of these methods to estimate the wind turbine power curve. Polynomial regression is studied as the benchmark parametric model, and issues associated with this technique are discussed. We then introduce the locally weighted polynomial regression method, and show its advantages over the polynomial regression. Also, the spline regression method is examined to achieve more flexibility for fitting the power curve. Finally, we develop a penalized spline regression model to address the issues of choosing the number and location of knots in the spline regression. The performance of the presented methods is evaluated using two simulated data sets as well as an actual operational power data of a wind farm in North America.

A Discretization-Free Sparse and Parametric Approach for Linear Array Signal Processing

Abstract: Direction of arrival (DOA) estimation in array pro- cessing using uniform/sparse linear arrays is concerned in this pa- per. While sparse methods via approximate parameter discretiza- tion have been popular in the past decade, the discretization may cause problems, e.g., modeling error and increased computations due to dense sampling. In this paper, an exact discretization- free method, named as sparse and parametric approach (SPA), is proposed for uniform and sparse linear arrays. SPA carries out parameter estimation in the continuous range based on well- established covariance fitting criteria and convex optimization. It guarantees to produce a sparse parameter estimate without discretization required by existing sparse methods. Theoretical analysis shows that the SPA parameter estimator is a large- snapshot realization of the maximum likelihood estimator and is statistically consistent (in the number of snapshots) under uncorrelated sources. Other merits of SPA include improved resolution, applicability to arbitrary number of snapshots, ro- bustness to correlation of the sources and no requirement of user-parameters. Numerical simulations are carried out to verify our analysis and demonstrate advantages of SPA compared to existing methods.

On-line learning of parametric mixture models for light transport simulation

Abstract: Monte Carlo techniques for light transport simulation rely on importance sampling when constructing light transport paths. Previous work has shown that suitable sampling distributions can be recovered from particles distributed in the scene prior to rendering. We propose to represent the distributions by a parametric mixture model trained in an on-line (i.e. progressive) manner from a potentially infinite stream of particles. This enables recovering good sampling distributions in scenes with complex lighting, where the necessary number of particles may exceed available memory. Using these distributions for sampling scattering directions and light emission significantly improves the performance of state-of-the-art light transport simulation algorithms when dealing with complex lighting.

Parametric and Nonparametric Statistical Methods for Genomic Selection of Traits with Additive and Epistatic Genetic Architectures

Abstract: Parametric and nonparametric methods have been developed for purposes of predicting phenotypes. These methods are based on retrospective analyses of empirical data consisting of genotypic and phenotypic scores. Recent reports have indicated that parametric methods are unable to predict phenotypes of traits with known epistatic genetic architectures. Herein, we review parametric methods including least squares regression, ridge regression, Bayesian ridge regression, least absolute shrinkage and selection operator (LASSO), Bayesian LASSO, best linear unbiased prediction (BLUP), Bayes A, Bayes B, Bayes C, and Bayes Cπ. We also review nonparametric methods including Nadaraya-Watson estimator, reproducing kernel Hilbert space, support vector machine regression, and neural networks. We assess the relative merits of these 14 methods in terms of accuracy and mean squared error (MSE) using simulated genetic architectures consisting of completely additive or two-way epistatic interactions in an F2 population derived from crosses of inbred lines. Each simulated genetic architecture explained either 30% or 70% of the phenotypic variability. The greatest impact on estimates of accuracy and MSE was due to genetic architecture. Parametric methods were unable to predict phenotypic values when the underlying genetic architecture was based entirely on epistasis. Parametric methods were slightly better than nonparametric methods for additive genetic architectures. Distinctions among parametric methods for additive genetic architectures were incremental. Heritability, i.e., proportion of phenotypic variability, had the second greatest impact on estimates of accuracy and MSE.

Applied Statistics Using SPSS, STATISTICA and MATLAB

Abstract: Introduction.- Presenting and Summarising the Data.- Estimating Data Parameters.- Parametric Tests of Hypotheses.- Non-Parametric Tests of Hypotheses.- Statistical Classification.- Data Regression.- Data Structure Analysis.- Directional Data.- Appendix A - Short Survey on Probability Theory.- Appendix B - Distributions.- Appendix C - Point Estimation.- Appendix E - Tables.- Appendix E - Datasets.- Appendix F - Tools.

Controlling the dynamic range of a Josephson parametric amplifier

Abstract: One of the central challenges in the development of parametric amplifiers is the control of the dynamic range relative to its gain and bandwidth, which typically limits quantum limited amplification to signals which contain only a few photons per inverse bandwidth. Here, we discuss the control of the dynamic range of Josephson parametric amplifiers by using Josephson junction arrays. We discuss gain, bandwidth, noise, and dynamic range properties of both a transmission line and a lumped element based parametric amplifier. Based on these investigations we derive useful design criteria, which may find broad application in the development of practical parametric amplifiers.

An instrumental variable approach for identification and estimation with nonignorable nonresponse

Abstract: Estimation based on data with nonignorable nonresponse is considered when the joint distribution of the study variable y and covariate x is nonpara- metric and the nonresponse probability conditional on y and x has a parametric form. The likelihood based on observed data may not be identifiable even when the joint distribution of y and x is parametric. We show that this difficulty can be overcome by utilizing a nonresponse instrument, an auxiliary variable related to y but not related to the nonresponse probability conditional on y and x. Under some conditions we can apply the generalized method of moments (GMM) to obtain es- timators of the parameters in the nonresponse probability and the nonparametric joint distribution of y and x. Consistency and asymptotic normality of GMM es- timators are established. Simulation results and an application to a data set from the Korean Labor and Income Panel Survey are also presented.

Comparison of parametric and non-parametric approaches for vehicle speed prediction

Abstract: Predicting the future speed of the ego-vehicle is a necessary component of many Intelligent Transportation Systems (ITS) applications, in particular for safety and energy management systems. In the last four decades many parametric speed prediction models have been proposed, the most advanced ones being developed for use in traffic simulators. More recently non-parametric approaches have been applied to closely related problems in robotics. This paper presents a comparative evaluation of parametric and non-parametric approaches for speed prediction during highway driving. Real driving data is used for the evaluation, and both short-term and long-term predictions are tested. The results show that the relative performance of the different models vary strongly with the prediction horizon. This should be taken into account when selecting a prediction model for a given ITS application.

Robust field-aligned global parametrization

Abstract: We present a robust method for computing locally bijective global parametrizations aligned with a given cross-field. The singularities of the parametrization in general agree with singularities of the field, except in a small number of cases when several additional cones need to be added in a controlled way. Parametric lines can be constrained to follow an arbitrary set of feature lines on the surface. Our method is based on constructing an initial quad patch partition using robust cross-field integral line tracing. This process is followed by an algorithm modifying the quad layout structure to ensure that consistent parametric lengths can be assigned to the edges. For most meshes, the layout modification algorithm does not add new singularities; a small number of singularities may be added to resolve an explicitly described set of layouts. We demonstrate that our algorithm succeeds on a test data set of over a hundred meshes.

Bayesian Nonparametric Poisson Factorization for Recommendation Systems

Abstract: We develop a Bayesian nonparametric Poisson factorization model for recommendation systems. Poisson factorization implicitly models each user’s limited budget of attention (or money) that allows consumption of only a small subset of the available items. In our Bayesian nonparametric variant, the number of latent components is theoretically unbounded and eectively estimated when computing a posterior with observed user behavior data. To approximate the posterior, we develop an ecient variational inference algorithm. It adapts the dimensionality of the latent components to the data, only requires iteration over the user/item pairs that have been rated, and has computational complexity on the same order as for a parametric model with xed dimensionality. We studied our model and algorithm with large realworld data sets of user-movie preferences. Our model eases the computational burden of searching for the number of latent components and gives better predictive performance than its parametric counterpart.

Model‐based dose finding under model uncertainty using general parametric models

Abstract: The statistical methodology for the design and analysis of clinical Phase II dose-response studies, with related software implementation, is well developed for the case of a normally distributed, homoscedastic response considered for a single timepoint in parallel group study designs. In practice, however, binary, count, or time-to-event endpoints are encountered, typically measured repeatedly over time and sometimes in more complex settings like crossover study designs. In this paper, we develop an overarching methodology to perform efficient multiple comparisons and modeling for dose finding, under uncertainty about the dose-response shape, using general parametric models. The framework described here is quite broad and can be utilized in situations involving for example generalized nonlinear models, linear and nonlinear mixed effects models, Cox proportional hazards models, with the main restriction being that a univariate dose-response relationship is modeled, that is, both dose and response correspond to univariate measurements. In addition to the core framework, we also develop a general purpose methodology to fit dose-response data in a computationally and statistically efficient way. Several examples illustrate the breadth of applicability of the results. For the analyses, we developed the R add-on package DoseFinding, which provides a convenient interface to the general approach adopted here. Copyright © 2013 John Wiley & Sons, Ltd.

Synchrony: quantifying variability in space and time

Abstract: Summary There is growing recognition that linking patterns to their underlying processes in interconnected and dynamic ecological systems requires data sampled at multiple spatial and temporal scales. However, spatially explicit and temporally resolved data sets can be difficult to analyze using classical statistical methods because the data are typically autocorrelated and thus violate the assumption of independence. Here, we describe the synchrony package for the R programming environment, which provides modern parametric and nonparametric methods for (i) quantifying temporal and spatial patterns of auto- and cross-correlated variability in univariate, bivariate, and multivariate data sets, and (ii) assessing their statistical significance via Monte Carlo randomizations. We illustrate how the methods included in the package can be used to investigate the causes of spatial and temporal variability in ecological systems through a series of examples, and discuss the assumptions and caveats of each statistical procedure in order to provide a practical guide for their application in the real world.

Nonparametric maximum likelihood approach to multiple change-point problems

Abstract: In multiple change-point problems, different data segments often follow different distributions, for which the changes may occur in the mean, scale or the entire distribution from one segment to another. Without the need to know the number of change-points in advance, we propose a nonparametric maximum likelihood approach to detecting multiple change-points. Our method does not impose any parametric assumption on the underlying distributions of the data sequence, which is thus suitable for detection of any changes in the distributions. The number of change-points is determined by the Bayesian information criterion and the locations of the change-points can be estimated via the dynamic programming algorithm and the use of the intrinsic order structure of the likelihood function. Under some mild conditions, we show that the new method provides consistent estimation with an optimal rate. We also suggest a prescreening procedure to exclude most of the irrelevant points prior to the implementation of the nonparametric likelihood method. Simulation studies show that the proposed method has satisfactory performance of identifying multiple change-points in terms of estimation accuracy and computation time.

Data-driven estimation in equilibrium using inverse optimization

Abstract: Equilibrium modeling is common in a variety of fields such as game theory and transportation science. The inputs for these models, however, are often difficult to estimate, while their outputs, i.e., the equilibria they are meant to describe, are often directly observable. By combining ideas from inverse optimization with the theory of variational inequalities, we develop an efficient, data-driven technique for estimating the parameters of these models from observed equilibria. We use this technique to estimate the utility functions of players in a game from their observed actions and to estimate the congestion function on a road network from traffic count data. A distinguishing feature of our approach is that it supports both parametric and nonparametric estimation by leveraging ideas from statistical learning (kernel methods and regularization operators). In computational experiments involving Nash and Wardrop equilibria in a nonparametric setting, we find that a) we effectively estimate the unknown demand or congestion function, respectively, and b) our proposed regularization technique substantially improves the out-of-sample performance of our estimators.

High‐resolution spectral characterization of two photon states via classical measurements

Abstract: Quantum optics plays a central role in the study of fundamental concepts in quantum mechanics, and in the development of new technological applications. Typical experiments employ sources of photon pairs generated by parametric processes such as spontaneous parametric down-conversion and spontaneous four-wave-mixing. The standard characterization of these sources relies on detecting the pairs themselves and thus requires single photon detectors, which limit both measurement speed and accuracy. Here it is shown that the two-photon quantum state that would be generated by parametric fluorescence can be characterised with unprecedented spectral resolution by performing a classical experiment. This streamlined technique gives access to hitherto unexplored features of two-photon states and has the potential to speed up design and testing of massively parallel integrated nonlinear sources by providing a fast and reliable quality control procedure. Additionally, it allows for the engineering of quantum light states at a significantly higher level of spectral detail, powering future quantum optical applications based on time-energy photon correlations.