scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Output-Sensitive Algorithms for Computing Nearest-Neighbour Decision Boundaries

TL;DR: Output-sensitive algorithms for computing this decision boundary for point sets on the line and in ℝ2 are developed, which is the best possible when parameterizing with respect to n and k.
Abstract: Given a set R of red points and a set B of blue points, the nearest-neighbour decision rule classifies a new point q as red (respectively, blue) if the closest point to q in R ⋃ B comes from R (respectively, B). This rule implicitly partitions space into a red set and a blue set that are separated by a red-blue decision boundary. In this paper we develop output-sensitive algorithms for computing this decision boundary for point sets on the line and in ℝ2. Both algorithms run in time O(n log k), where k is the number of points that contribute to the decision boundary. This running time is the best possible when parameterizing with respect to n and k.

Content maybe subject to copyright    Report

Citations
More filters
Journal ArticleDOI
TL;DR: The proposed method incorporates the voting method into the popular extreme learning machine (ELM) in classification applications and generally outperforms the original ELM algorithm as well as several recent classification algorithms.

329 citations


Cites methods from "Output-Sensitive Algorithms for Com..."

  • ...In this subsection, simulations using V-ELM, SVM [12], OP-ELM [28], BP [9,24,27], and KNN [7,2] are conducted on all the above 19 datasets....

    [...]

  • ...For KNN, 7 nearest neighbors are used and the Euclidean norm is adopted to calculate the distance....

    [...]

  • ...Simulations on many real world classification datasets demonstrate that V-ELM outperforms several recent methods in general, including the original ELM [14], support vector machine (SVM) [12], optimally pruned extreme learning machine (OP-ELM) [28], Back-Propagation algorithm (BP) [9,24,27], K nearest neighbors algorithm (KNN) [2,7], robust fuzzy relational classifier (RFRC) [5], radial basis function neural network (RBFNN) [33] and multiobjective simultaneous learning framework (MSCC) [6]....

    [...]

  • ...In this section, the performance of the proposed V-ELM is compared with the original ELM [14], SVM [12] and several other recent classification methods, including OP-ELM [28], Back-Propagation algorithm (BP) [9,24,27], K nearest neighbors algorithm (KNN) [2,7], RFRC [5], RBFNN [33] and MSCC [6]....

    [...]

  • ...It is easy to find from this table that with K = 7, V-ELM learns much faster than SVM and BP for all the datasets, faster than OP-ELM in all the datasets except for Iris, Monk1 and Monk2, and faster than KNN in all the datasets except for Soybean, Hayes and Protein....

    [...]

Journal ArticleDOI
TL;DR: These effects represent the first use of resting-state functional network connectivity (FNC) features to classify schizophrenia and show that classification with high accuracy is achievable using simple non-linear discriminative methods such as k-nearest neighbors (KNNs) which is very promising.
Abstract: There is a growing interest in automatic classification of mental disorders based on neuroimaging data. Small training data sets (subjects) and very large amount of high dimensional data make it a challenging task to design robust and accurate classifiers for heterogeneous disorders such as schizophrenia. Most previous studies considered structural MRI, diffusion tensor imaging and task-based fMRI for this purpose. However, resting-state data has been rarely used in discrimination of schizophrenia patients from healthy controls. Resting data are of great interest, since they are relatively easy to collect, and not confounded by behavioral performance on a task. Several linear and non-linear classification methods were trained using a training dataset and evaluate with a separate testing dataset. Results show that classification with high accuracy is achievable using simple non-linear discriminative methods such as k-nearest neighbors which is very promising. We compare and report detailed results of each classifier as well as statistical analysis and evaluation of each single feature. To our knowledge our effects represent the first use of resting-state functional network connectivity features to classify schizophrenia.

178 citations

Journal ArticleDOI
TL;DR: The importance of native biodiversity in buffering the impacts of climate change is demonstrated, because crop pollination services would decline more steeply without the native, wild pollinators, which is an important example of how biodiversity can stabilize ecosystem services against environmental change.
Abstract: If climate change affects pollinator-dependent crop production, this will have important implications for global food security because insect pollinators contribute to production for 75% of the leading global food crops. We investigate whether climate warming could result in indirect impacts upon crop pollination services via an overlooked mechanism, namely temperature-induced shifts in the diurnal activity patterns of pollinators. Using a large data set on bee pollination of watermelon crops, we predict how pollination services might change under various climate change scenarios. Our results show that under the most extreme IPCC scenario (A1F1), pollination services by managed honey bees are expected to decline by 14.5%, whereas pollination services provided by most native, wild taxa are predicted to increase, resulting in an estimated aggregate change in pollination services of +4.5% by 2099. We demonstrate the importance of native biodiversity in buffering the impacts of climate change, because crop pollination services would decline more steeply without the native, wild pollinators. More generally, our study provides an important example of how biodiversity can stabilize ecosystem services against environmental change.

157 citations


Cites methods from "Output-Sensitive Algorithms for Com..."

  • ...To generate a mean and a variance for the flower visitation rate within each cell of the temperature-time grid, we used the k-nearest neighbour algorithm (Bremner et al., 2005)....

    [...]

  • ...To generate a mean and a variance for the flower visitation rate within each cell of the temperature-time grid, we used the k-nearest neighbour algorithm (Bremner et al., 2005)....

    [...]

Proceedings Article
25 Jan 2012
TL;DR: A general Bag of Words model is used in order to compare two different classification methods, both K-Nearest-Neighbor and Support-Vector-Machine, and it is observed that the SVM classifier outperformed the KNN classifier.
Abstract: In order for a robot or a computer to perform tasks, it must recognize what it is looking at. Given an image a computer must be able to classify what the image represents. While this is a fairly simple task for humans, it is not an easy task for computers. Computers must go through a series of steps in order to classify a single image. In this paper, we used a general Bag of Words model in order to compare two different classification methods. Both K-Nearest-Neighbor (KNN) and Support-Vector-Machine (SVM) classification are well known and widely used. We were able to observe that the SVM classifier outperformed the KNN classifier. For future work, we hope to use more categories for the objects and to use more sophisticated classifiers.

122 citations

Journal ArticleDOI
TL;DR: In this paper, a method for analysing the spatially coincident multibeam bathymetric and backscatter data from shallow coastal waters to generate spatial data products that relate to the classes derived from fine-scale visual imagery obtained using an autonomous underwater vehicle (AUV).
Abstract: In this study we outline the techniques used to transform multibeam acoustic data into spatial layers that can be used for predictive habitat modelling. The results allow us to identify multibeam attributes which may act as potential surrogates for environmental variables that influence biodiversity and define which variables may be reliable for predicting the distribution of species in temperate waters. We explore a method for analysing the spatially coincident multibeam bathymetric and backscatter data from shallow coastal waters to generate spatial data products that relate to the classes derived from fine-scale visual imagery obtained using an autonomous underwater vehicle (AUV). Classifications of the multibeam data are performed for substrate, rugosity and sponge cover. Overall classification accuracies for the classes associated with substratum, rugosity and sponge structure were acceptable for biodiversity assessment applications. Accuracies were highest for rugosity classes at 65%, followed by substratum classes at 64% and then sponge structure classes at 57%. Random forest classifiers at a segmentation scale of 30 performed best in classifying substratum and rugosity, while K-nearest neighbour classifiers performed best for sponge structure classes, with no difference in accuracy between scale 30 and 60. Incorporating backscatter variables using segmentation improved the overall accuracy achieved by the best performing model by between 1% (rugosity) and 9% (substratum) above using topographic variables only in the grid-based analyses. Results suggest that image-based backscatter classification show considerable promise for the interpretation of multibeam sonar data for the production of substrate maps. A particular outcome of this research is to provide appropriate and sufficiently fine-scale physical covariates from the multibeam acoustic data to adequately inform models predicting the distribution of biodiversity on benthic reef habitats.

121 citations

References
More filters
Journal ArticleDOI
TL;DR: The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points, so it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.
Abstract: The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points. This rule is independent of the underlying joint distribution on the sample points and their classifications, and hence the probability of error R of such a rule must be at least as great as the Bayes probability of error R^{\ast} --the minimum probability of error over all decision rules taking underlying probability structure into account. However, in a large sample analysis, we will show in the M -category case that R^{\ast} \leq R \leq R^{\ast}(2 --MR^{\ast}/(M-1)) , where these bounds are the tightest possible, for all suitably smooth underlying distributions. Thus for any number of categories, the probability of error of the nearest neighbor rule is bounded above by twice the Bayes probability of error. In this sense, it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.

12,243 citations

Journal ArticleDOI
TL;DR: In this article, a sequence of probability weight functions defined in terms of nearest neighbors is constructed and sufficient conditions for consistency are obtained, which are applied to verify the consistency of the estimators of the various quantities discussed above and the consistency in Bayes risk of the approximate Bayes rules.
Abstract: Let $(X, Y)$ be a pair of random variables such that $X$ is $\mathbb{R}^d$-valued and $Y$ is $\mathbb{R}^{d'}$-valued. Given a random sample $(X_1, Y_1), \cdots, (X_n, Y_n)$ from the distribution of $(X, Y)$, the conditional distribution $P^Y(\bullet \mid X)$ of $Y$ given $X$ can be estimated nonparametrically by $\hat{P}_n^Y(A \mid X) = \sum^n_1 W_{ni}(X)I_A(Y_i)$, where the weight function $W_n$ is of the form $W_{ni}(X) = W_{ni}(X, X_1, \cdots, X_n), 1 \leqq i \leqq n$. The weight function $W_n$ is called a probability weight function if it is nonnegative and $\sum^n_1 W_{ni}(X) = 1$. Associated with $\hat{P}_n^Y(\bullet \mid X)$ in a natural way are nonparametric estimators of conditional expectations, variances, covariances, standard deviations, correlations and quantiles and nonparametric approximate Bayes rules in prediction and multiple classification problems. Consistency of a sequence $\{W_n\}$ of weight functions is defined and sufficient conditions for consistency are obtained. When applied to sequences of probability weight functions, these conditions are both necessary and sufficient. Consistent sequences of probability weight functions defined in terms of nearest neighbors are constructed. The results are applied to verify the consistency of the estimators of the various quantities discussed above and the consistency in Bayes risk of the approximate Bayes rules.

1,754 citations


"Output-Sensitive Algorithms for Com..." refers background in this paper

  • ...Several properties make the nearest-neighbour decision rule quite attractive, including its intuitive simplicity and the theorem that the asymptotic error rate of the nearestneighbour rule is bounded from above by twice the Bayes error rate [6], [8], [16]....

    [...]

Journal ArticleDOI
TL;DR: The number of comparisons required to select the i-th smallest of n numbers is shown to be at most a linear function of n by analysis of a new selection algorithm-PICK.

1,384 citations


"Output-Sensitive Algorithms for Com..." refers methods in this paper

  • ...To find the decision boundary in O(n log k) time, we begin by computing the median element m = s� n/2 in O(n) time using any one of the existing linear-time median finding algorithms (see [ 3 ])....

    [...]

Journal ArticleDOI
TL;DR: In this paper, it was shown that for nonparametric estimators of a density function, the Taylor polynomial is the optimal (uniform) rate of convergence for a sequence of estimators.
Abstract: Let $d$ denote a positive integer, $\|x\| = (x^2_1 + \cdots + x^2_d)^{1/2}$ the Euclidean norm of $x = (x_1, \cdots, x_d) \in \mathbb{R}^d, k$ a nonnegative integer, $\mathscr{C}_k$ the collection of $k$ times continuously differentiable functions on $\mathbb{R}^d$, and $g_k$ the Taylor polynomial of degree $k$ about the origin corresponding to $g \in \mathscr{C}_k$. Let $M$ and $p > k$ denote positive constants and let $U$ be an open neighborhood of the origin of $\mathbb{R}^d$. Let $\mathscr{G}$ denote the collection of functions $g \in \mathscr{C}_k$ such that $|g(x) - g_k(x)| \leq M \|x\|^P$ for $x\in U$. Let $m \leq k$ be a nonnegative integer, let $\theta_0\in\mathscr{C}_m$ and set $\Theta = \{\theta_0 + g:g \in \mathscr{G}\}$. Let $L$ be a linear differential operator of order $m$ on $\mathscr{C}_m$ and set $T(\theta) = L\theta(0)$ for $\theta \in \Theta$. Let $(X, Y)$ be a pair of random variables such that $X$ is $\mathbb{R}^d$ valued and $Y$ is real valued. It is assumed that the distribution of $X$ is absolutely continuous and that its density is bounded away from zero and infinity on $U$. The conditional distribution of $Y$ given $X$ is assumed to be (say) normal, with a conditional variance which is bounded away from zero and infinity on $U$. The regression function of $Y$ on $X$ is assumed to belong to $\Theta$. It is shown that $r = (p - m)/(2p + d)$ is the optimal (uniform) rate of convergence for a sequence $\{\hat{T}_n\}$ of estimators of $T(\theta)$ such that $\hat{T}_n$ is based on a random sample of size $n$ from the distribution of $(X, Y)$. An analogous result is obtained for nonparametric estimators of a density function.

837 citations

Journal ArticleDOI
TL;DR: This work presents a practical algorithm for subdivision search that achieves the same (optimal) worst case complexity bounds as the significantly more complex algorithm of Lipton and Tarjan, namely $O(\log n)$ search time with $O(n)$ storage.
Abstract: A planar subdivision is any partition of the plane into (possibly unbounded) polygonal regions. The subdivision search problem is the following: given a subdivision $S$ with $n$ line segments and a query point $p$, determine which region of $S$ contains $p$. We present a practical algorithm for subdivision search that achieves the same (optimal) worst case complexity bounds as the significantly more complex algorithm of Lipton and Tarjan, namely $O(\log n)$ search time with $O(n)$ storage. Our subdivision search structure can be constructed in linear time from the subdivision representation used in many applications.

810 citations


"Output-Sensitive Algorithms for Com..." refers methods in this paper

  • ...Alternatively, the algorithm we describe for computing the nearest-neighbour decision boundary actually produces the Vorono˘ i diagram of the condensed set (which has size O(k)) that can be preprocessed in O(k) time by Kirkpatrick’s point-location algorithm [ 12 ] to allow nearest-neighbour classification in O(log k) time....

    [...]