Data Mining - Concepts and Techniques.

1. Introduction 1.1 An Overview 1.2 Some Examples 1.3 A Brief History 1.4 A Chapter Summary 2. Probability 2.1 Introduction 2.2 Sample Spaces and the Algebra of Sets 2.3 The Probability Function 2.4 Conditional Probability 2.5 Independence 2.6 Combinatorics 2.7 Combinatorial Probability 2.8 Taking a Second Look at Statistics (Monte Carlo Techniques) 3. Random Variables 3.1 Introduction 3.2 Binomial and Hypergeometric Probabilities 3.3 Discrete Random Variables 3.4 Continuous Random Variables 3.5 Expected Values 3.6 The Variance 3.7 Joint Densities 3.8 Transforming and Combining Random Variables 3.9 Further Properties of the Mean and Variance 3.10 Order Statistics 3.11 Conditional Densities 3.12 Moment-Generating Functions 3.13 Taking a Second Look at Statistics (Interpreting Means) Appendix 3.A.1 MINITAB Applications 4. Special Distributions 4.1 Introduction 4.2 The Poisson Distribution 4.3 The Normal Distribution 4.4 The Geometric Distribution 4.5 The Negative Binomial Distribution 4.6 The Gamma Distribution 4.7 Taking a Second Look at Statistics (Monte Carlo Simulations) Appendix 4.A.1 MINITAB Applications Appendix 4.A.2 A Proof of the Central Limit Theorem 5. Estimation 5.1 Introduction 5.2 Estimating Parameters: The Method of Maximum Likelihood and the Method of Moments 5.3 Interval Estimation 5.4 Properties of Estimators 5.5 Minimum-Variance Estimators: The Crami?½r-Rao Lower Bound 5.6 Sufficient Estimators 5.7 Consistency 5.8 Bayesian Estimation 5.9 Taking A Second Look at Statistics (Beyond Classical Estimation) Appendix 5.A.1 MINITAB Applications 6. Hypothesis Testing 6.1 Introduction 6.2 The Decision Rule 6.3 Testing Binomial Dataâ H0: p = po 6.4 Type I and Type II Errors 6.5 A Notion of Optimality: The Generalized Likelihood Ratio 6.6 Taking a Second Look at Statistics (Statistical Significance versus â Practicalâ Significance) 7. Inferences Based on the Normal Distribution 7.1 Introduction 7.2 Comparing Y-i?½ s/ vn and Y-i?½ S/ vn 7.3 Deriving the Distribution of Y-i?½ S/ vn 7.4 Drawing Inferences About i?½ 7.5 Drawing Inferences About s2 7.6 Taking a Second Look at Statistics (Type II Error) Appendix 7.A.1 MINITAB Applications Appendix 7.A.2 Some Distribution Results for Y and S2 Appendix 7.A.3 A Proof that the One-Sample t Test is a GLRT Appendix 7.A.4 A Proof of Theorem 7.5.2 8. Types of Data: A Brief Overview 8.1 Introduction 8.2 Classifying Data 8.3 Taking a Second Look at Statistics (Samples Are Not â Validâ !) 9. Two-Sample Inferences 9.1 Introduction 9.2 Testing H0: i?½X =i?½Y 9.3 Testing H0: s2X=s2Yâ The F Test 9.4 Binomial Data: Testing H0: pX = pY 9.5 Confidence Intervals for the Two-Sample Problem 9.6 Taking a Second Look at Statistics (Choosing Samples) Appendix 9.A.1 A Derivation of the Two-Sample t Test (A Proof of Theorem 9.2.2) Appendix 9.A.2 MINITAB Applications 10. Goodness-of-Fit Tests 10.1 Introduction 10.2 The Multinomial Distribution 10.3 Goodness-of-Fit Tests: All Parameters Known 10.4 Goodness-of-Fit Tests: Parameters Unknown 10.5 Contingency Tables 10.6 Taking a Second Look at Statistics (Outliers) Appendix 10.A.1 MINITAB Applications 11. Regression 11.1 Introduction 11.2 The Method of Least Squares 11.3 The Linear Model 11.4 Covariance and Correlation 11.5 The Bivariate Normal Distribution 11.6 Taking a Second Look at Statistics (How Not to Interpret the Sample Correlation Coefficient) Appendix 11.A.1 MINITAB Applications Appendix 11.A.2 A Proof of Theorem 11.3.3 12. The Analysis of Variance 12.1 Introduction 12.2 The F Test 12.3 Multiple Comparisons: Tukeyâ s Method 12.4 Testing Subhypotheses with Contrasts 12.5 Data Transformations 12.6 Taking a Second Look at Statistics (Putting the Subject of Statistics togetherâ the Contributions of Ronald A. Fisher) Appendix 12.A.1 MINITAB Applications Appendix 12.A.2 A Proof of Theorem 12.2.2 Appendix 12.A.3 The Distribution of SSTR/(kâ 1) SSE/(nâ k)When H1 is True 13. Randomized Block Designs 13.1 Introduction 13.2 The F Test for a Randomized Block Design 13.3 The Paired t Test 13.4 Taking a Second Look at Statistics (Choosing between a Two-Sample t Test and a Paired t Test) Appendix 13.A.1 MINITAB Applications 14. Nonparametric Statistics 14.1 Introduction 14.2 The Sign Test 14.3 Wilcoxon Tests 14.4 The Kruskal-Wallis Test 14.5 The Friedman Test 14.6 Testing for Randomness 14.7 Taking a Second Look at Statistics (Comparing Parametric and Nonparametric Procedures) Appendix 14.A.1 MINITAB Applications Appendix: Statistical Tables Answers to Selected Odd-Numbered Questions Bibliography Index

http://library.um.ac.id/free-contents/downloadpdf.php/buku/an-introduction-to-mathematical-statistics-and-its-applications-richard-j-larsen-morris-l-marx-12385.pdf

An introduction to mathematical statistical and its applications / Richard J. Larsen, Morris L. Marx

Mining outliers in database is to find exceptional objects that deviate from the rest of the data set. Besides classical outlier analysis algorithms, recent studies have focused on mining local outliers, i.e., the outliers that have density distribution significantly different from their neighborhood. The estimation of density distribution at the location of an object has so far been based on the density distribution of its k-nearest neighbors [2,11]. However, when outliers are in the location where the density distributions in the neighborhood are significantly different, for example, in the case of objects from a sparse cluster close to a denser cluster, this may result in wrong estimation. To avoid this problem, here we propose a simple but effective measure on local outliers based on a symmetric neighborhood relationship. The proposed measure considers both neighbors and reverse neighbors of an object when estimating its density distribution. As a result, outliers so discovered are more meaningful. To compute such local outliers efficiently, several mining algorithms are developed that detects top-n outliers based on our definition. A comprehensive performance evaluation and analysis shows that our methods are not only efficient in the computation but also more effective in ranking outliers.

Ranking outliers using symmetric neighborhood relationship

The moderate resolution imaging spectroradiometer (MODIS) flown on-board NASA's first earth observing system (EOS) platform, Terra, offers complete global data coverage every 1–2 days at spatial resolutions of 250, 500, and 1000 m. Its ability to detect emitted radiation in the short (4 μm)- and long (12 μm)-wave infrared regions of the electromagnetic spectrum, combined with the excellent geolocation of the image pixels (∼200 m), makes it an ideal source of data for automatically detecting and monitoring high-temperature volcanic thermal anomalies. This paper describes the underlying principles of, and results obtained from, just such a system. Our algorithm interrogates the MODIS Level 1B data stream for evidence of high-temperature volcanic features. Once a hotspot has been identified, its details (location, emitted spectral radiance, satellite observational parameters) are written to an ASCII text file and transferred via file transfer protocol (FTP) to the Hawaii Institute of Geophysics and Planetology (HIGP), where the results are posted on the Internet ( http://modis.higp.hawaii.edu ). The global distribution of volcanic hotspots can be examined visually at a variety of scales using this website, which also allows easy access to the quantitative data contained in the ASCII files themselves. We outline how the algorithm has proven robust as a hotspot detection tool for a wide range of eruptive styles at both permanently and sporadically active volcanoes including Soufriere Hills (Montserrat), Popocatepetl (Mexico), Bezymianny (Russia), and Merapi (Java), amongst others. We also present case studies of how the system has allowed the onset, development, and cessation of discrete eruptive events to be monitored at Nyamuragira (Congo), Piton de la Fournaise (Reunion Island), and Shiveluch (Russia).

Automated Volcanic Eruption Detection Using MODIS

A new density-based clustering algorithm, RNN-DBSCAN , is presented which uses reverse nearest neighbor counts as an estimate of observation density. Clustering is performed using a DBSCAN -like approach based on $k$ nearest neighbor graph traversals through dense observations. RNN-DBSCAN is preferable to the popular density-based clustering algorithm DBSCAN in two aspects. First, problem complexity is reduced to the use of a single parameter (choice of $k$ nearest neighbors), and second, an improved ability for handling large variations in cluster density (heterogeneous density). The superiority of RNN-DBSCAN is demonstrated on several artificial and real-world datasets with respect to prior work on reverse nearest neighbor based clustering approaches ( RECORD , IS-DBSCAN , and ISB-DBSCAN ) along with DBSCAN and OPTICS . Each of these clustering approaches is described by a common graph-based interpretation wherein clusters of dense observations are defined as connected components, along with a discussion on their computational complexity. Heuristics for RNN-DBSCAN parameter selection are presented, and the effects of $k$ on RNN-DBSCAN clusterings discussed. Additionally, with respect to scalability, an approximate version of RNN-DBSCAN is presented leveraging an existing approximate $k$ nearest neighbor technique.

RNN-DBSCAN: A Density-Based Clustering Algorithm Using Reverse Nearest Neighbor Density Estimates

© 2012 Cassisi et al., licensee InTech. This is an open access chapter distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Similarity Measures and Dimensionality Reduction Techniques for Time Series Data Mining

/pdf/similarity-measures-and-dimensionality-reduction-techniques-cgx02ef9ez.pdf

Similarity Measures and Dimensionality Reduction Techniques for Time Series Data Mining

Enhancing density-based clustering: Parameter reduction and outlier detection

Volcanoes generate a broad range of seismo-volcanic and infrasonic signals, whose features and variations are often closely related to volcanic activity. The study of these signals is hence very useful in the monitoring and investigation of volcano dynamics. The analysis of seismo-volcanic and infrasonic signals requires specifically developed techniques due to their unique characteristics, which are generally quite distinct compared with tectonic and volcano-tectonic earthquakes. In this work, we describe analysis methods used to detect and locate seismo-volcanic and infrasonic signals at Mt. Etna. Volcanic tremor sources are located using a method based on spatial seismic amplitude distribution, assuming propagation in a homogeneous medium. The tremor source is found by calculating the goodness of the linear regression fit (R
 2) of the log-linearized equation of the seismic amplitude decay with distance. The location method for long-period events is based on the joint computation of semblance and R
 2 values, and the location method of very long-period events is based on the application of radial semblance. Infrasonic events and tremor are located by semblance–brightness- and semblance-based methods, respectively. The techniques described here can also be applied to other volcanoes and do not require particular network geometries (such as arrays) but rather simple sparse networks. Using the source locations of all the considered signals, we were able to reconstruct the shallow plumbing system (above sea level) during 2011.

Monitoring Seismo-volcanic and Infrasonic Signals at Volcanoes: Mt. Etna Case Study

SUMMARY Active volcanoes generate sonic and infrasonic signals, whose investigation provides useful information for both monitoring purposes and the study of the dynamics of explosive phenomena. At Mt. Etna volcano (Italy), a pattern recognition system based on infrasonic waveform features has been developed. First, by a parametric power spectrum method, the features describing and characterizing the infrasound events were extracted: peak frequency and quality factor. Then, together with the peak-to-peak amplitude, these features constituted a 3-D ‘feature space’; by Density-Based Spatial Clustering of Applications with Noise algorithm (DBSCAN) three clusters were recognized inside it. After the clustering process, by using a common location method (semblance method) and additional volcanological information concerning the intensity of the explosive activity, we were able to associate each cluster to a particular source vent and/or a kind of volcanic activity. Finally, for automatic event location, clusters were used to train a model based on Support Vector Machine, calculating optimal hyperplanes able to maximize the margins of separation among the clusters. After the training phase this system automatically allows recognizing the active vent with no location algorithm and by using only a single station.

/pdf/clustering-and-classification-of-infrasonic-events-at-mount-2pb2d9wu3f.pdf

Clustering and classification of infrasonic events at Mount Etna using pattern recognition techniques

In this work, we present a new catalog of focal mechanisms calculated for earthquakes recorded in Sicily and southern Calabria. It comprises about 300 solutions, for events with magnitudes ranging from 2.7 to 4.8 that occurred from 1999 to 2011. We used P-wave polarities to compute the fault-plane solutions. Two main goals are achieved. For the first, the catalog allows the stress regime and kinematics characterizing the studied area to be depicted at a regional and more local scale. In particular, moving along the tectonic lineament that extends from the Aeolian Islands to the Ionian Sea, there is a change from a regime characterized by sub-horizontal P-axes, ca. NW-SE directed, to an extensive one in the Calabro-Peloritan Arc, where T-axes striking in a NW-SE direction prevail. Our results also show that part of the seismicity is clustered along the main active seismogenic structures, of which the focal mechanisms indicate the kinematics. Finally, in the Etna volcano area, different stress fields act at different depths due to the combination of the regional tectonics, the strong pressurization of the deep magmatic system, and the dynamics of the shallower portion of the volcano. As a second goal, we highlight that the catalog also represents a valuable tool, through the data distribution on the internet, for further studies directed towards improving our understanding of the geodynamic complexity of the region, and for a better characterization of the seismogenic sources.

/pdf/sicily-and-southern-calabria-focal-mechanism-database-a-4ltjx4ry99.pdf

Carmelo Cassisi

Papers

Similarity Measures and Dimensionality Reduction Techniques for Time Series Data Mining

Enhancing density-based clustering: Parameter reduction and outlier detection

Monitoring Seismo-volcanic and Infrasonic Signals at Volcanoes: Mt. Etna Case Study

Clustering and classification of infrasonic events at Mount Etna using pattern recognition techniques

Sicily and southern Calabria focal mechanism database: a valuable tool for local and regional stress-field determination