The increasing volume of data in modern business and science calls for more complex and sophisticated tools. Although advances in data mining technology have made extensive data collection much easier, it's still always evolving and there is a constant need for new techniques and tools that can help us transform this data into useful information and knowledge. Since the previous edition's publication, great advances have been made in the field of data mining. Not only does the third of edition of Data Mining: Concepts and Techniques continue the tradition of equipping you with an understanding and application of the theory and practice of discovering patterns hidden in large data sets, it also focuses on new, important topics in the field: data warehouses and data cube technology, mining stream, mining social networks, and mining spatial, multimedia and other complex data. Each chapter is a stand-alone guide to a critical topic, presenting proven algorithms and sound implementations ready to be used directly or with strategic modification against live data. This is the resource you need if you want to apply today's most powerful data mining techniques to meet real business challenges. * Presents dozens of algorithms and implementation examples, all in pseudo-code and suitable for use in real-world, large-scale data mining projects. * Addresses advanced topics such as mining object-relational databases, spatial databases, multimedia databases, time-series databases, text databases, the World Wide Web, and applications in several fields. *Provides a comprehensive, practical look at the concepts and techniques you need to get the most out of real business data

/pdf/data-mining-concepts-and-techniques-4dtvdfkvmi.pdf

Data Mining: Concepts and Techniques

Statistics for Spatial Data.

The authors propose a new climatic drought index: the standardized precipitation evapotranspiration index (SPEI). The SPEI is based on precipitation and temperature data, and it has the advantage of combining multiscalar character with the capacity to include the effects of temperature variability on drought assessment. The procedure to calculate the index is detailed and involves a climatic water balance, the accumulation of deficit/surplus at different time scales, and adjustment to a log-logistic probability distribution. Mathematically, the SPEI is similar to the standardized precipitation index (SPI), but it includes the role of temperature. Because the SPEI is based on a water balance, it can be compared to the self-calibrated Palmer drought severity index (sc-PDSI). Time series of the three indices were compared for a set of observatories with different climate characteristics, located in different parts of the world. Under global warming conditions, only the sc-PDSI and SPEI identified an...

/pdf/a-multiscalar-drought-index-sensitive-to-global-warming-the-370wre0ti5.pdf

A Multiscalar Drought Index Sensitive to Global Warming: The Standardized Precipitation Evapotranspiration Index

During the late 1990s, Huang introduced the algorithm called Empirical Mode Decomposition, which is widely used today to recursively decompose a signal into different modes of unknown but separate spectral bands. EMD is known for limitations like sensitivity to noise and sampling. These limitations could only partially be addressed by more mathematical attempts to this decomposition problem, like synchrosqueezing, empirical wavelets or recursive variational decomposition. Here, we propose an entirely non-recursive variational mode decomposition model, where the modes are extracted concurrently. The model looks for an ensemble of modes and their respective center frequencies, such that the modes collectively reproduce the input signal, while each being smooth after demodulation into baseband. In Fourier domain, this corresponds to a narrow-band prior. We show important relations to Wiener filter denoising. Indeed, the proposed method is a generalization of the classic Wiener filter into multiple, adaptive bands. Our model provides a solution to the decomposition problem that is theoretically well founded and still easy to understand. The variational model is efficiently optimized using an alternating direction method of multipliers approach. Preliminary results show attractive performance with respect to existing mode decomposition models. In particular, our proposed model is much more robust to sampling and noise. Finally, we show promising practical decomposition results on a series of artificial and real data.

Variational Mode Decomposition

/pdf/statistical-analysis-in-climate-research-51ik9f04rx.pdf

Statistical Analysis in Climate Research

L-moments are expectations of certain linear combinations of order statistics. They can be defined for any random variable whose mean exists and form the basis of a general theory which covers the summarization and description of theoretical probability distributions, the summarization and description of observed data samples, estimation of parameters and quantiles of probability distributions, and hypothesis tests for probability distributions. The theory involves such established procedures as the use of order statistics and Gini's mean difference statistic, and gives rise to some promising innovations such as the measures of skewness and kurtosis and new methods of parameter estimation

https://www.jstor.org/stable/info/2345653

L-Moments: Analysis and Estimation of Distributions Using Linear Combinations of Order Statistics

Preface 1. Regional frequency analysis 2. L-moments 3. Screening the data 4. Identification of homogeneous regions 5. Choice of a frequency distribution 6. Estimation of the frequency distribution 7. Performance of the regional L-moment algorithm 8. Other topics 9. Examples Appendix References Index of notation.

Regional Frequency Analysis: An Approach Based on L-Moments

We use the method of probability-weighted moments to derive estimators of the parameters and quantiles of the generalized extreme-value distribution. We investigate the properties of these estimators in large samples, via asymptotic theory, and in small and moderate samples, via computer simulation. Probability-weighted moment estimators have low variance and no severe bias, and they compare favorably with estimators obtained by the methods of maximum likelihood or sextiles. The method of probability-weighted moments also yields a convenient and powerful test of whether an extreme-value distribution is of Fisher-Tippett Type I, II, or III.

Estimation of the generalized extreme-value distribution by the method of probability-weighted moments

The generalized Pareto distribution is a two-parameter distribution that contains uniform, exponential, and Pareto distributions as special cases. It has applications in a number of fields, including reliability studies and the analysis of environmental extreme events. Maximum likelihood estimation of the generalized Pareto distribution has previously been considered in the literature, but we show, using computer simulation, that, unless the sample size is 500 or more, estimators derived by the method of moments or the method of probability-weighted moments are more reliable. We also use computer simulation to assess the accuracy of confidence intervals for the parameters and quantiles of the generalized Pareto distribution.

Parameter and quantile estimation for the generalized pareto distribution

Regional frequency analysis uses data from a number of measuring sites. A “region” is a group of sites each of which is assumed to have data drawn from the same frequency distribution. The analysis involves the assignment of sites to regions, testing whether the proposed regions are indeed homogeneous, and choice of suitable distributions to fit to each region's data. This paper describes three statistics useful in regional frequency analysis: a discordancy measure, for identifying unusual sites in a region; a heterogeneity measure, for assessing whether a proposed region is homogeneous; and a goodness-of-fit measure, for assessing whether a candidate distribution provides an adequate fit to the data. Tests based on the statistics provide objective backing for the decisions involved in regional frequency analysis. The statistics are based on the L moments [Hosking, 1990] of the at-site data.

Jonathan R. M. Hosking

Papers

L-Moments: Analysis and Estimation of Distributions Using Linear Combinations of Order Statistics

Regional Frequency Analysis: An Approach Based on L-Moments

Estimation of the generalized extreme-value distribution by the method of probability-weighted moments

Parameter and quantile estimation for the generalized pareto distribution

Some statistics useful in regional frequency analysis