Optimum experimental designs

Summary One attractive feature of optimum design criteria, such as D- and A-optimality, is that they are directly related to statistically interpretable properties of the designs that are obtained, such as minimizing the volume of a joint confidence region for the parameters However, the assumed relationships with inferential procedures are valid only if the variance of experimental units is assumed to be known If the variance is estimated, then the properties of the inferences depend also on the number of degrees of freedom that are available for estimating the error variance Modified optimality criteria are defined, which correctly reflect the utility of designs with respect to some common types of inference For fractional factorial and response surface experiments, the designs that are obtained are quite different from those which are optimal under the standard criteria, with many more replicate points required to estimate error The optimality of these designs assumes that inference is the only purpose of running the experiment, but in practice interpretation of the point estimates of parameters and checking for lack of fit of the treatment model assumed are also usually important Thus, a compromise between the new criteria and others is likely to be more relevant to many practical situations Compound criteria are developed, which take account of multiple objectives, and are applied to fractional factorial and response surface experiments The resulting designs are more similar to standard designs but still have sufficient residual degrees of freedom to allow effective inferences to be carried out The new procedures developed are applied to three experiments from the food industry to see how the designs used could have been improved and to several illustrative examples The design optimization is implemented through a simple exchange algorithm

Optimum design of experiments for statistical inference

that the design and implementation stages of statistical graphics play a very important role in dealing successfully with the challenges arising from large datasets. Chapter 4 concentrates on the application of interactive graphics to large datasets to show how they can help improve the power and potential of graphics in supplying more information by employing some sophisticated techniques such as querying, scaling, linking, zooming, and sorting. This chapter ends with some remarks about what should be done to improve interaction capabilities of existing data analysis tools when dealing with large and complex datasets. Chapters 5–11 that are on particular types of graphics for applications constitute the second part of the book titled as “Applications.” Chapter 5 examines the mosaic plots and their variations for multivariate categorical data involving both large numbers of cases and large numbers of categories. This chapter also demonstrates how valuable it is to use techniques like reordering and redmarking to produce clearer images of what is in the data. Chapter 6 concentrates on useful visualization techniques, known as rotating plots, that can be used to explore multivariate continuous data, and overviews the software developed and used for this purpose. Chapter 7 provides a very nice introduction to a smooth modified version of the parallel coordinate plot that uses color brushing to separate categories and can easily handle many observations in more than three dimensions. This chapter contains some mathematics to explain the logic behind the new modified parallel plot and shows by an example the role some sophisticated actions such as reordering and rescaling of axes can play in obtaining useful representations of large datasets. Chapter 8 examines the networks by concentrating on the difference between drawing optimal layouts and producing informative displays within a reasonable time limit. Because of focusing on the difference between time and quality, the emphasis in this chapter is on exploration and discovery that let the reader to reveal unusual features hidden within very large graphs by using a variety of layout algorithms. Chapter 9 examines trees that are known to be useful for exploring crucial patterns in the small datasets. This chapter generalizes the logic behind the trees to the large datasets by focusing on the task of combining and summarizing the information obtained from large numbers of trees complex datasets can produce. The behavior of some innovative visualization methods, such as fluctuation diagrams and treemaps, to get more information from large datasets is considered in detail in this chapter. Chapter 10 concentrates on some challenging visualization problems related to the “Mice and Elephants” graphic by applying it to the internet packet data. The methods offered in this chapter, such as biased sampling and quantile windows, to deal with the visualizations of large and complex datasets are shown to uncover some interesting structures in the internet traffic dataset. The last chapter, Chapter 11, puts the visualization methods described in the previous chapters of the book into action using a real dataset to illustrate how valuable visualization can be in the process of data analysis. This is a valuable book for all the researchers who need practical guidance to explore their large datasets by the help of a variety of visualization methods. Several applications discussed throughout the book and easy-to-read explanations of the methods make them easy to apply for anyone. We recommend this book for everyone interested in discovering structures hidden in the large datasets by a variety of state-of-the-art visualization techniques. Since the authors use the word “a million” throughout the book as a “useful symbolic target” for large datasets, we also think that it is going to be useful to read the third page of the book to learn what Francis Galton thought more than a century ago about what a million might look like.

A Modern Theory of Factorial Design

Network method of moments arXiv:1202.5101 is an important tool for nonparametric network inferences. However, there has been little investigation on accurate descriptions of the sampling distributions of network moment statistics. In this paper, we present the first higher-order accurate approximation to the sampling CDF of a studentized network moment by Edgeworth expansion. In sharp contrast to classical literature on noiseless U-statistics, we showed that the Edgeworth expansion of a network moment statistic as a noisy U-statistic can achieve higher-order accuracy without non-lattice or smoothness assumptions but just requiring weak regularity conditions. Behind this result is our surprising discovery that the two typically-hated factors in network analysis, namely, sparsity and edge-wise observational errors, jointly play a blessing role, contributing a crucial self-smoothing effect in the network moment statistic and making it analytically tractable. Our assumptions match the minimum requirements in related literature. 
For practitioners, our empirical Edgeworth expansion is highly accurate and computationally efficient. It is also easy to implement. These were demonstrated by comprehensive simulation studies. 
We showcase three applications of our results in network inference. We proved, to our knowledge, for the first time that some network bootstraps enjoy higher-order accuracy, and provided theoretical guidance for tuning network sub-sampling. We also derived a one-sample test and Cornish-Fisher confidence interval for any given moment, both with analytical formulation and explicit error rates.

Edgeworth expansions for network moments

In this paper we consider nonlinear models with an arbitrary number of covariates for which the information additionally depends on the value of the linear predictor. We establish the general result that for many optimality criteria the support points of an optimal design lie on the edges of the design region, if this design region is a polyhedron. Based on this result we show that under certain conditions the D-optimal designs can be constructed from the D-optimal designs in the marginal models with single covariates. This can be applied to a broad class of models, which include the Poisson, the negative binomial as well as the proportional hazards model with both type I and random censoring.

Optimal Design for Multiple Regression with Information Driven by the Linear Predictor

Sequential experiments composed of initial experiments and follow-up experiments are widely adopted for economical computer emulations. Many kinds of Latin hypercube designs with good space-filling properties have been proposed for designing the initial computer experiments. However, little work based on Latin hypercubes has focused on the design of the follow-up experiments. Although some constructions of nested Latin hypercube designs can be adapted to sequential designs, the size of the follow-up experiments needs to be a multiple of that of the initial experiments. In this article, a general method for constructing sequential designs of flexible size is proposed, which allows the combined designs to have good one-dimensional space-filling properties. Moreover, the sampling properties and a type of central limit theorem are derived for these designs. Several improvements of these designs are made to achieve better space-filling properties. Simulations are carried out to verify the theoretical r...

Design for Sequential Follow-Up Experiments in Computer Emulations

Sliced Latin hypercube designs are popularly adopted for computer experiments with qualitative factors. Previous constructions require the sizes of different slices to be identical. Here we construct sliced designs with flexible sizes of slices. Besides achieving desirable one-dimensional uniformity, flexible sliced designs (FSDs) constructed in this paper accommodate arbitrary sizes for different slices and cover ordinary sliced Latin hypercube designs as special cases. The sampling properties of FSDs are derived and a central limit theorem is established. It shows that any linear combination of the sample means from different models on slices follows an asymptotic normal distribution. Some simulations compare FSDs with other sliced designs in collective evaluations of multiple computer models.

Flexible sliced designs for computer experiments

U-statistics are widely used in fields such as economics, machine learning, and statistics. However, while they enjoy desirable statistical properties, they have an obvious drawback in that the computation becomes impractical as the data size $n$ increases. Specifically, the number of combinations, say $m$, that a U-statistic of order $d$ has to evaluate is $O(n^d)$. Many efforts have been made to approximate the original U-statistic using a small subset of combinations since Blom (1976), who referred to such an approximation as an incomplete U-statistic. To the best of our knowledge, all existing methods require $m$ to grow at least faster than $n$, albeit more slowly than $n^d$, in order for the corresponding incomplete U-statistic to be asymptotically efficient in terms of the mean squared error. In this paper, we introduce a new type of incomplete U-statistic that can be asymptotically efficient, even when $m$ grows more slowly than $n$. In some cases, $m$ is only required to grow faster than $\sqrt{n}$. Our theoretical and empirical results both show significant improvements in the statistical efficiency of the new incomplete U-statistic.

/pdf/design-based-incomplete-u-statistics-11v07a1wd4.pdf

Design based incomplete u-statistics

The Lasso approach is widely adopted for screening and estimating active effects in sparse linear models with quantitative factors. Many design schemes have been proposed based on different criteria to make the Lasso estimator more accurate. This article applies $$\varPhi _l$$-optimality to the asymptotic covariance matrix of the Lasso estimator. Smaller mean squared error and higher power of significant hypothesis tests can be achieved. A theoretically converging algorithm is given for searching for $$\varPhi _l$$-optimal designs, and modified by intermittent diffusion to avoid local solutions. Some simulations are given to support the theoretical results.

Xiangshun Kong

Papers

Design for Sequential Follow-Up Experiments in Computer Emulations

Flexible sliced designs for computer experiments

Design based incomplete u-statistics

Optimal designs in sparse linear models

Optimal designs for dose–response models with linear effects of covariates