http://midag.cs.unc.edu/pubs/papers/Neuroimage06_Yushkevich.pdf

User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability

In k-means clustering, we are given a set of n data points in d-dimensional space R/sup d/ and an integer k and the problem is to determine a set of k points in Rd, called centers, so as to minimize the mean squared distance from each data point to its nearest center. A popular heuristic for k-means clustering is Lloyd's (1982) algorithm. We present a simple and efficient implementation of Lloyd's k-means clustering algorithm, which we call the filtering algorithm. This algorithm is easy to implement, requiring a kd-tree as the only major data structure. We establish the practical efficiency of the filtering algorithm in two ways. First, we present a data-sensitive analysis of the algorithm's running time, which shows that the algorithm runs faster as the separation between clusters increases. Second, we present a number of empirical studies both on synthetically generated data and on real data sets from applications in color quantization, data compression, and image segmentation.

/pdf/an-efficient-k-means-clustering-algorithm-analysis-and-1vgn3zr7cg.pdf

An efficient k-means clustering algorithm: analysis and implementation

Gradient Boosting Decision Tree (GBDT) is a popular machine learning algorithm, and has quite a few effective implementations such as XGBoost and pGBRT. Although many engineering optimizations have been adopted in these implementations, the efficiency and scalability are still unsatisfactory when the feature dimension is high and data size is large. A major reason is that for each feature, they need to scan all the data instances to estimate the information gain of all possible split points, which is very time consuming. To tackle this problem, we propose two novel techniques: Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB). With GOSS, we exclude a significant proportion of data instances with small gradients, and only use the rest to estimate the information gain. We prove that, since the data instances with larger gradients play a more important role in the computation of information gain, GOSS can obtain quite accurate estimation of the information gain with a much smaller data size. With EFB, we bundle mutually exclusive features (i.e., they rarely take nonzero values simultaneously), to reduce the number of features. We prove that finding the optimal bundling of exclusive features is NP-hard, but a greedy algorithm can achieve quite good approximation ratio (and thus can effectively reduce the number of features without hurting the accuracy of split point determination by much). We call our new GBDT implementation with GOSS and EFB LightGBM. Our experiments on multiple public datasets show that, LightGBM speeds up the training process of conventional GBDT by up to over 20 times while achieving almost the same accuracy.

/pdf/lightgbm-a-highly-efficient-gradient-boosting-decision-tree-f835dtmi7b.pdf

LightGBM: a highly efficient gradient boosting decision tree

Today there are several eAE ient algorithms that ope with the popular and omputationally expensive task of asso iation rule mining. A tually, these algorithms are more or less des ribed on their own. In this paper we explain the fundamentals of asso iation rule mining and moreover derive a general framework. Based on this we des ribe today's approa hes in ontext by pointing out ommon aspe ts and di eren es. After that we thoroughly investigate their strengths and weaknesses and arry out several runtime experiments. It turns out that the runtime behavior of the algorithms is mu h more similar as to be expe ted.

Algorithms for association rule mining — a general survey and comparison

The k-means algorithm is by far the most widely used method for discovering clusters in data. We show how to accelerate it dramatically, while still always computing exactly the same result as the standard algorithm. The accelerated algorithm avoids unnecessary distance calculations by applying the triangle inequality in two different ways, and by keeping track of lower and upper bounds for distances between points and centers. Experiments show that the new algorithm is effective for datasets with up to 1000 dimensions, and becomes more and more effective as the number k of clusters increases. For k ≥ 20 it is many times faster than the best previously known accelerated k-means method.

/pdf/using-the-triangle-inequality-to-accelerate-k-means-3ui2kxu0xp.pdf

Using the triangle inequality to accelerate k-means

In this paper, we present a novel algorithm for performing k-means clustering. It organizes all the patterns in a k-d tree structure such that one can find all the patterns which are closest to a given prototype efficiently. The main intuition behind our approach is as follows. All the prototypes are potential candidates for the closest prototype at the root level. However, for the children of the root node, we may be able to prune the candidate set by using simple geometrical constraints. This approach can be applied recursively until the size of the candidate set is one for each node. Our experimental results demonstrate that our scheme can improve the computational speed of the direct k-means algorithm by an order to two orders of magnitude in the total number of distance calculations and the overall time of computation.

/pdf/an-efficient-k-means-clustering-algorithm-bqqp3dzg85.pdf

An efficient k-means clustering algorithm

Efficient discovery of association rules in large databases is a well studied problem and several approaches have been proposed. However, it is non trivial to maintain the association rules current when the database is updated since, such updates could invalidate existing rules or introduce new rules. In this paper, we propose an incremental updating technique based on negative borders, for the maintenance of association rules when new transaction data is added to or deleted from a transaction database. An important feature of our algorithm is that it requires a full scan (exactly one) of the whole database only if the database update causes the negative border of the set of large itemsets to expand.

/pdf/an-efficient-algorithm-for-the-incremental-updation-of-2msaoi19uq.pdf

An efficient algorithm for the incremental updation of association rules in large databases

Classification for very large datasets has many practical applications in data mining. Techniques such as discretization and dataset sampling can be used to scale up decision tree classifiers to large datasets. Unfortunately, both of these techniques can cause a significant loss in accuracy. We present a novel decision tree classifier called CLOUDS, which samples the splitting points for numeric attributes followed by an estimation step to narrow the search space of the best split. CLOUDS reduces computation and I/O complexity substantially compared to state of the art classifiers, while maintaining the quality of the generated trees in terms of accuracy and tree size. We provide experimental results with a number of real and synthetic datasets.

/pdf/clouds-a-decision-tree-classifier-for-large-datasets-1z4yw0qm80.pdf

CLOUDS: a decision tree classifier for large datasets

The present invention is directed to an improved data clustering method and apparatus for use in data mining operations. The present invention determines the pattern vectors of a k-d tree structure which are closest to a given prototype cluster by pruning prototypes through geometrical constraints, before a k-means process is applied to the prototypes. For each sub-branch in the k-d tree, a candidate set of prototypes is formed from the parent of a child node. The minimum and maximum distances from any point in the child node to any prototype in the candidate set is determined. The smallest of the maximum distances found is compared to the minimum distances of each prototype in the candidate set. Those prototypes with a minimum distance greater than the smallest of the maximum distances are pruned or eliminated. Pruning the number of remote prototypes reduces the number of distance calculations for the k-means process, significantly reducing the overall computation time.

Method and apparatus for reducing the computational requirements of K-means data clustering

The cpquantile of an ordered sequence of data values is the element with rank ‘pn, where n is the total number of values. Accurate estimates of quantiles are required for the solution of many practical problems. In this paper, we present a new algorithm for estimating the quantile values for disk-resident data. Our algorithm has the following characteristics: (1) It requires only one pass over the data; (2) It is deterministic; (3) It produces good lower and upper bounds of the true values of the quantiles; (4) It requires no a priori knowledge of the distribution of the data set; (5) It has a scalable parallel formulation; (6) Extra time and memory for computing additional quantiles (beyond the first one) are constant per quantile. We present experimental results on the IBM SP-2. The experimental results show that the algorithm is indeed robust and does not depend on the distribution of the data sets.

/pdf/a-one-pass-algorithm-for-accurately-estimating-quantiles-for-1hoa9j7no4.pdf

Khaled Alsabti

Papers

An efficient k-means clustering algorithm

An efficient algorithm for the incremental updation of association rules in large databases

CLOUDS: a decision tree classifier for large datasets

Method and apparatus for reducing the computational requirements of K-means data clustering

A One-Pass Algorithm for Accurately Estimating Quantiles for Disk-Resident Data