Reproducibility is essential to reliable scientific discovery in high-throughput experiments. In this work we propose a unified approach to measure the reproducibility of findings identified from replicate experiments and identify putative discoveries using reproducibility. Unlike the usual scalar measures of reproducibility, our approach creates a curve, which quantitatively assesses when the findings are no longer consistent across replicates. Our curve is fitted by a copula mixture model, from which we derive a quantitative reproducibility score, which we call the "irreproducible discovery rate" (IDR) analogous to the FDR. This score can be computed at each set of paired replicate ranks and permits the principled setting of thresholds both for assessing reproducibility and combining replicates. Since our approach permits an arbitrary scale for each replicate, it provides useful descriptive measures in a wide variety of situations to be explored. We study the performance of the algorithm using simulations and give a heuristic analysis of its theoretical properties. We demonstrate the effectiveness of our method in a ChIP-seq experiment.

Measuring reproducibility of high-throughput experiments

[1] This article presents the modeling of multivariate extreme values using copulas. Our approach allows us to model the dependence structure independently of the marginal distributions, which is not possible with standard classical methods. The methodology has been applied on two different problems in hydrology. The first application is concerned with the combined risk in the framework of frequency analysis. Four copulas have been tested on peak flows from the watershed of Peribonka in Quebec, Canada. The second application relates to the joint modeling of peak flows and volumes. Three copulas have been applied to the watershed of the Rimouski River in Quebec, Canada. This approach using copulas is promising since it allows us to take into account a wide range of correlation which can happen in hydrology.

/pdf/multivariate-hydrological-frequency-analysis-using-copulas-586e5velea.pdf

Multivariate hydrological frequency analysis using copulas

Ranked lists are encountered in research and daily life and it is often of interest to compare these lists even when they are incomplete or have only some members in common. An example is document rankings returned for the same query by different search engines. A measure of the similarity between incomplete rankings should handle nonconjointness, weight high ranks more heavily than low, and be monotonic with increasing depth of evaluation; but no measure satisfying all these criteria currently exists. In this article, we propose a new measure having these qualities, namely rank-biased overlap (RBO). The RBO measure is based on a simple probabilistic user model. It provides monotonicity by calculating, at a given depth of evaluation, a base score that is non-decreasing with additional evaluation, and a maximum score that is nonincreasing. An extrapolated score can be calculated between these bounds if a point estimate is required. RBO has a parameter which determines the strength of the weighting to top ranks. We extend RBO to handle tied ranks and rankings of different lengths. Finally, we give examples of the use of the measure in comparing the results produced by public search engines and in assessing retrieval systems in the laboratory.

A similarity measure for indefinite rankings

There is a growing interest in the development of natural fiber-reinforced composites, most likely due to their wide availability, low cost, environment friendliness, and sustainability. The market...

Hydrophobic treatment of natural fibers and their composites—A review

Resin transfer molding of natural fiber reinforced composites: cure simulation

This paper deals with the modelling and simulation of resin flow, heat transfer and the curing of multilayer thermoset composite laminates during processing in an autoclave. Darcy's Law and Stokes’ slow-flow equations are used for the flow model and, for approximately isothermal flows, a similarity solution is developed. This permits the decoupling of the velocity and thermal fields. A two-dimensional convection–diffusion heat equation with an internal heat generation term is then solved numerically, together with the equation for the rate of cure, using a finite difference scheme on a moving grid. The simulations are performed with varying composite thicknesses, and a comparison of numerical results with known experimental data confirms the approximate validity of the model.

Curing simulation of thermoset composites

Within the bounds of a general theory of rank correlation two particular measures have been adopted widely: Spearman's rank correlation coefficient, ρ, in which ranks replace variates in Pearson's product-moment correlation calculation; and Kendall's τ, in which the disarray of x-ordered data due to a y-ordering is measured by counting the minimum number, s, of transpositions (interchanges between adjacent ranks) of the y-ordering sufficient to recover the x-ordering. Based on insights from the calculation of Kendall's coefficient, this paper develops a graphical approach which leads to a new rank correlation coefficient akin to that of Spearman. This measure appears to stand outside general theory but has greater power of discrimination amongst differing reorderings of the data whilst simultaneously being strongly correlated with both ρ and τ. The development is focused on situations where agreement over ordering is more important for top place getters than for those lower down the order as, for example, in subjectively judged Olympic events such as ice skating. The basic properties of the proposed coefficient are identified. © Australian Statistical Publishing Association Inc. 2000. Published by Blackwell Publishers Ltd.

Rank correlation - an alternative measure

Curing simulation by autoclave resin infusion

This paper analyses a set of world records in athletic running events to extract long-term bounds for those events. The approach adopted is to identify a single parameter to represent the achieved standard of athletic performance at a series of fixed intervals. The long-term behaviour of this single parameter is then investigated by fitting a variety of non-linear models and restrictions on the accuracy of the fits are discussed. The paper concludes with a range of estimates for each of the events considered in the original data set.

Lower bounds for athletic performance

Studies of kurtosis often concentrate on only symmetric distributions. This paper identifies a process through which the standardized measure of kurtosis based on the fourth moment about the mean can be written in terms of two parts: (i) an irreducible component, about L4, which can be seen to occur naturally in the analysis of fourth moments; (ii) terms that depend only on moments of lower order, in particular including the effects of asymmetry attached to the third moment about the mean. This separation of the effect of skewness allows definition of an improved measure of kurtosis. This paper calculates and discusses examples of the new measure of kurtosis for a range of standard distributions.

DC Blest

Papers

Curing simulation of thermoset composites

Rank correlation - an alternative measure

Curing simulation by autoclave resin infusion

Lower bounds for athletic performance

A New Measure of Kurtosis Adjusted for Skewness