The increasing volume of data in modern business and science calls for more complex and sophisticated tools. Although advances in data mining technology have made extensive data collection much easier, it's still always evolving and there is a constant need for new techniques and tools that can help us transform this data into useful information and knowledge. Since the previous edition's publication, great advances have been made in the field of data mining. Not only does the third of edition of Data Mining: Concepts and Techniques continue the tradition of equipping you with an understanding and application of the theory and practice of discovering patterns hidden in large data sets, it also focuses on new, important topics in the field: data warehouses and data cube technology, mining stream, mining social networks, and mining spatial, multimedia and other complex data. Each chapter is a stand-alone guide to a critical topic, presenting proven algorithms and sound implementations ready to be used directly or with strategic modification against live data. This is the resource you need if you want to apply today's most powerful data mining techniques to meet real business challenges. * Presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects. * Addresses advanced topics such as mining object-relational databases, spatial databases, multimedia databases, time-series databases, text databases, the World Wide Web, and applications in several fields. *Provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data

/pdf/data-mining-concepts-and-techniques-4dtvdfkvmi.pdf

Data Mining: Concepts and Techniques

This book describes the new generation of discrete choice methods, focusing on the many advances that are made possible by simulation. Researchers use these statistical methods to examine the choices that consumers, households, firms, and other agents make. Each of the major models is covered: logit, generalized extreme value, or GEV (including nested and cross-nested logits), probit, and mixed logit, plus a variety of specifications that build on these basics. Simulation-assisted estimation procedures are investigated and compared, including maximum simulated likelihood, method of simulated moments, and method of simulated scores. Procedures for drawing from densities are described, including variance reduction techniques such as anithetics and Halton draws. Recent advances in Bayesian procedures are explored, including the use of the Metropolis-Hastings algorithm and its variant Gibbs sampling. No other book incorporates all these fields, which have arisen in the past 20 years. The procedures are applicable in many fields, including energy, transportation, environmental studies, health, labor, and marketing.

/pdf/discrete-choice-methods-with-simulation-l1x4cghjtt.pdf

Discrete Choice Methods with Simulation

Many scientific phenomena are now investigated by complex computer models or codes A computer experiment is a number of runs of the code with various inputs A feature of many computer experiments is that the output is deterministic--rerunning the code with the same inputs gives identical observations Often, the codes are computationally expensive to run, and a common objective of an experiment is to fit a cheaper predictor of the output to the data Our approach is to model the deterministic output as the realization of a stochastic process, thereby providing a statistical basis for designing experiments (choosing the inputs) for efficient prediction With this model, estimates of uncertainty of predictions are also available Recent work in this area is reviewed, a number of applications are discussed, and we demonstrate our methodology with an example

/pdf/the-design-and-analysis-of-computer-experiments-1kfo89lyf2.pdf

The design and analysis of computer experiments

Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

The finite mixture (FM) model is the most commonly used model for statistical segmentation of brain magnetic resonance (MR) images because of its simple mathematical form and the piecewise constant nature of ideal brain MR images. However, being a histogram-based model, the FM has an intrinsic limitation-no spatial information is taken into account. This causes the FM model to work only on well-defined images with low levels of noise; unfortunately, this is often not the the case due to artifacts such as partial volume effect and bias field distortion. Under these conditions, FM model-based methods produce unreliable results. Here, the authors propose a novel hidden Markov random field (HMRF) model, which is a stochastic process generated by a MRF whose state sequence cannot be observed directly but which can be indirectly estimated through observations. Mathematically, it can be shown that the FM model is a degenerate version of the HMRF model. The advantage of the HMRF model derives from the way in which the spatial information is encoded through the mutual influences of neighboring sites. Although MRF modeling has been employed in MR image segmentation by other researchers, most reported methods are limited to using MRF as a general prior in an FM model-based approach. To fit the HMRF model, an EM algorithm is used. The authors show that by incorporating both the HMRF model and the EM algorithm into a HMRF-EM framework, an accurate and robust segmentation can be achieved. More importantly, the HMRF-EM framework can easily be combined with other techniques. As an example, the authors show how the bias field correction algorithm of Guillemaud and Brady (1997) can be incorporated into this framework to achieve a three-dimensional fully automated approach for brain MR image segmentation.

/pdf/segmentation-of-brain-mr-images-through-a-hidden-markov-2vxcvxn8aq.pdf

Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm

Two convergence aspects of the EM algorithm are studied: (i) does the EM algorithm find a local maximum or a stationary value of the (incomplete-data) likelihood function? (ii) does the sequence of parameter estimates generated by EM converge? Several convergence results are obtained under conditions that are applicable to many practical situations Two useful special cases are: (a) if the unobserved complete-data specification can be described by a curved exponential family with compact parameter space, all the limit points of any EM sequence are stationary points of the likelihood function; (b) if the likelihood function is unimodal and a certain differentiability condition is satisfied, then any EM sequence converges to the unique maximum likelihood estimate A list of key properties of the algorithm is included

/pdf/on-the-convergence-properties-of-the-em-algorithm-zxjq0mqgzw.pdf

On the convergence properties of the em algorithm

Basic Principles and Experiments with a Single Factor. Experiments With More Than One Factor. Full Factorial Experiments at Two Levels. Fractional Factorial Experiments at Two Levels. Full Factorial and Fractional Factorial Experiments at Three Levels. Other Design and Analysis Techniques for Experiments at More Than Two Levels. Nonregular Designs: Construction and Properties. Experiments with Complex Aliasing. Response Surface Methodology. Introduction to Robust Parameter Design. Robust Parameter Design for Signal-Response Systems. Experiments for Improving Reliability. Experiments With Nonnormal Data. Appendices. Indexes.

Experiments: Planning, Analysis, and Parameter Design Optimization

Preface to the Second Edition. Preface to the First Edition. Suggestions of Topics for Instructors. List of Experiments and Data Sets. 1 Basic Concepts for Experimental Design and Introductory Regression Analysis. 1.1 Introduction and Historical Perspective. 1.2 A Systematic Approach to the Planning and Implementation of Experiments. 1.3 Fundamental Principles: Replication, Randomization, and Blocking. 1.4 Simple Linear Regression. 1.5 Testing of Hypothesis and Interval Estimation. 1.6 Multiple Linear Regression. 1.7 Variable Selection in Regression Analysis. 1.8 Analysis of Air Pollution Data. 1.9 Practical Summary. 2 Experiments with a Single Factor. 2.1 One-Way Layout. 2.2 Multiple Comparisons. 2.3 Quantitative Factors and Orthogonal Polynomials. 2.4 Expected Mean Squares and Sample Size Determination. 2.5 One-Way Random Effects Model. 2.6 Residual Analysis: Assessment of Model Assumptions. 2.7 Practical Summary. 3 Experiments with More Than One Factor. 3.1 Paired Comparison Designs. 3.2 Randomized Block Designs. 3.3 Two-Way Layout: Factors With Fixed Levels. 3.4 Two-Way Layout: Factors With Random Levels. 3.5 Multi-Way Layouts. 3.6 Latin Square Designs: Two Blocking Variables. 3.7 Graeco-Latin Square Designs. 3.8 Balanced Incomplete Block Designs. 3.9 Split-Plot Designs. 3.10 Analysis of Covariance: Incorporating Auxiliary Information. 3.11 Transformation of the Response. 3.12 Practical Summary. 4 Full Factorial Experiments at Two Levels. 4.1 An Epitaxial Layer Growth Experiment. 4.2 Full Factorial Designs at Two Levels: A General Discussion. 4.3 Factorial Effects and Plots. 4.4 Using Regression to Compute Factorial Effects. 4.5 ANOVA Treatment of Factorial Effects. 4.6 Fundamental Principles for Factorial Effects: Effect Hierarchy, Effect Sparsity, and Effect Heredity. 4.7 Comparisons with the "One-Factor-at-a-Time" Approach. 4.8 Normal and Half-Normal Plots for Judging Effect Significance. 4.9 Lenth's Method: Testing Effect Significance for Experiments Without Variance Estimates. 4.10 Nominal-the-Best Problem and Quadratic Loss Function. 4.11 Use of Log Sample Variance for Dispersion Analysis. 4.12 Analysis of Location and Dispersion: Revisiting the Epitaxial Layer Growth Experiment. 4.13 Test of Variance Homogeneity and Pooled Estimate of Variance. 4.14 Studentized Maximum Modulus Test: Testing Effect Significance for Experiments with Variance Estimates. 4.15 Blocking and Optimal Arrangement of 2 k Factorial Designs in 2 q Blocks. 4.16 Practical Summary. 5 Fractional Factorial Experiments at Two Levels. 5.1 A Leaf Spring Experiment. 5.2 Fractional Factorial Designs: Effect Aliasing and the Criteria Of Resolution and Minimum Aberration. 5.3 Analysis of Fractional Factorial Experiments. 5.4 Techniques for Resolving the Ambiguities in Aliased Effects. 5.5 Selection of 2 k-p Designs Using Minimum Aberration and Related Criteria. 5.6 Blocking in Fractional Factorial Designs. 5.7 Practical Summary. 6 Full Factorial and Fractional Factorial Experiments at Three Levels. 6.1 A Seat-Belt Experiment. 6.2 Larger-the-Better and Smaller-the-Better Problems. 6.3 3 k Full Factorial Designs. 6.4 3 k-p Fractional Factorial Designs. 6.5 Simple Analysis Methods: Plots and Analysis of Variance. 6.6 An Alternative Analysis Method. 6.7 Analysis Strategies for Multiple Responses I: Out-of-Spec Probabilities. 6.8 Blocking in 3 k and 3 k-p Designs. 6.9 Practical Summary. 7 Other Design and Analysis Techniques for Experiments at More Than Two Levels. 7.1 A Router Bit Experiment Based on a Mixed Two-Level and Four-Level Design. 7.2 Method of Replacement and Construction of 2 m 4 n Designs. 7.3 Minimum Aberration 2 m 4 n Designs with n = 1, 2. 7.4 An Analysis Strategy for 2 m 4 n Experiments. 7.5 Analysis of the Router Bit Experiment. 7.6 A Paint Experiment Based on a Mixed Two-Level and Three-Level Design. 7.7 Design and Analysis of 36-Run Experiments at Two And Three Levels. 7.8 r k-p Fractional Factorial Designs for any Prime Number r . 7.9 Related Factors: Method of Sliding Levels, Nested Effects Analysis, and Response Surface Modeling. 7.10 Practical Summary. 8 Nonregular Designs: Construction and Properties. 8.1 Two Experiments: Weld-Repaired Castings and Blood Glucose Testing. 8.2 Some Advantages of Nonregular Designs Over the 2 k-p and 3 k-p Series of Designs. 8.3 A Lemma on Orthogonal Arrays. 8.4 Plackett-Burman Designs and Hall's Designs. 8.5 A Collection of Useful Mixed-Level Orthogonal Arrays. 8.6 Construction of Mixed-Level Orthogonal Arrays Based on Difference Matrices. 8.7 Construction of Mixed-Level Orthogonal Arrays Through the Method of Replacement. 8.8 Orthogonal Main-Effect Plans Through Collapsing Factors. 8.9 Practical Summary. 9 Experiments with Complex Aliasing. 9.1 Partial Aliasing of Effects and the Alias Matrix. 9.2 Traditional Analysis Strategy: Screening Design and Main Effect Analysis. 9.3 Simplification of Complex Aliasing via Effect Sparsity. 9.4 An Analysis Strategy for Designs with Complex Aliasing. 9.5 A Bayesian Variable Selection Strategy for Designs with Complex Aliasing. 9.6 Supersaturated Designs: Design Construction and Analysis. 9.7 Practical Summary. 10 Response Surface Methodology. 10.1 A Ranitidine Separation Experiment. 10.2 Sequential Nature of Response Surface Methodology. 10.3 From First-Order Experiments to Second-Order Experiments: Steepest Ascent Search and Rectangular Grid Search. 10.4 Analysis of Second-Order Response Surfaces. 10.5 Analysis of the Ranitidine Experiment. 10.6 Analysis Strategies for Multiple Responses II: Contour Plots and the Use of Desirability Functions. 10.7 Central Composite Designs. 10.8 Box-Behnken Designs and Uniform Shell Designs. 10.9 Practical Summary. 11 Introduction to Robust Parameter Design. 11.1 A Robust Parameter Design Perspective of the Layer Growth and Leaf Spring Experiments. 11.2 Strategies for Reducing Variation. 11.3 Noise (Hard-to-Control) Factors. 11.4 Variation Reduction Through Robust Parameter Design. 11.5 Experimentation and Modeling Strategies I: Cross Array. 11.6 Experimentation and Modeling Strategies II: Single Array and Response Modeling. 11.7 Cross Arrays: Estimation Capacity and Optimal Selection. 11.8 Choosing Between Cross Arrays and Single Arrays. 11.9 Signal-to-Noise Ratio and Its Limitations for Parameter Design Optimization. 11.10 Further Topics. 11.11 Practical Summary. 12 Robust Parameter Design for Signal-Response Systems. 12.1 An Injection Molding Experiment. 12.2 Signal-Response Systems and their Classification. 12.3 Performance Measures for Parameter Design Optimization. 12.4 Modeling and Analysis Strategies. 12.5 Analysis of the Injection Molding Experiment. 12.6 Choice of Experimental Plans. 12.7 Practical Summary. 13 Experiments for Improving Reliability. 13.1 Experiments with Failure Time Data. 13.2 Regression Model for Failure Time Data. 13.3 A Likelihood Approach for Handling Failure Time Data with Censoring. 13.4 Design-Dependent Model Selection Strategies. 13.5 A Bayesian Approach to Estimation and Model Selection for Failure Time Data. 13.6 Analysis of Reliability Experiments with Failure Time Data. 13.7 Other Types of Reliability Data. 13.8 Practical Summary. 14 Analysis of Experiments with Nonnormal Data. 14.1 A Wave Soldering Experiment with Count Data. 14.2 Generalized Linear Models. 14.3 Likelihood-Based Analysis of Generalized Linear Models. 14.4 Likelihood-Based Analysis of the Wave Soldering Experiment. 14.5 Bayesian Analysis of Generalized Linear Models. 14.6 Bayesian Analysis of the Wave Soldering Experiment. 14.7 Other Uses and Extensions of Generalized Linear Models and Regression Models for Nonnormal Data. 14.8 Modeling and Analysis for Ordinal Data. 14.9 Analysis of Foam Molding Experiment. 14.10 Scoring: A Simple Method for Analyzing Ordinal Data. 14.11 Practical Summary. Appendix A Upper Tail Probabilities of the Standard Normal Distribution. Appendix B Upper Percentiles of the t Distribution. Appendix C Upper Percentiles of the chi 2 Distribution. Appendix D Upper Percentiles of the F Distribution. Appendix E Upper Percentiles of the Studentized Range Distribution. Appendix F Upper Percentiles of the Studentized Maximum Modulus Distribution. Appendix G Coefficients of Orthogonal Contrast Vectors. Appendix H Critical Values for Lenth's Method. Author Index. Subject Index.

Experiments : planning, analysis, and optimization

Supplementary Material: Sequential Exploration of Complex Surfaces Using Minimum Energy Designs

In the quest for advanced propulsion and power-generation systems, high-fidelity simulations are too computationally expensive to survey the desired design space, and a new design methodology is needed that combines engineering physics, computer simulations, and statistical modeling. In this article, we propose a new surrogate model that provides efficient prediction and uncertainty quantification of turbulent flows in swirl injectors with varying geometries, devices commonly used in many engineering applications. The novelty of the proposed method lies in the incorporation of known physical properties of the fluid flow as simplifying assumptions for the statistical model. In view of the massive simulation data at hand, which is on the order of hundreds of gigabytes, these assumptions allow for accurate flow predictions in around an hour of computation time. To contrast, existing flow emulators which forgo such simplifications may require more computation time for training and prediction than is n...

C. F. Jeff Wu

Papers

On the convergence properties of the em algorithm

Experiments: Planning, Analysis, and Parameter Design Optimization

Experiments : planning, analysis, and optimization

Supplementary Material: Sequential Exploration of Complex Surfaces Using Minimum Energy Designs

An Efficient Surrogate Model for Emulation and Physics Extraction of Large Eddy Simulations